Contributing
Due to the precarious nature of the Open Specifications Promise, it is very important to ensure code is cleanroom. Contribution Notes
File organization (click to show)
Folders:
| folder | contents |
|---|---|
bin | server-side bin scripts (xlsx.njs) |
bits | raw source files that make up the final script |
dist | dist files for web browsers and nonstandard JS environments |
misc | miscellaneous supporting scripts |
modules | TypeScript source files that generate some of the bits |
packages | Support libraries and tools |
test_files | test files (pulled from the test files repository) |
tests | browser tests (run make ctest to rebuild) |
types | TypeScript definitions and tests |
After cloning the repo, running make help will display a list of commands.
Setup
These instructions will cover system configuration, cloning the source repo, building, reproducing official releases, and running NodeJS and browser tests.
These instructions were tested on the following platforms:
| Platform | Test Date |
|---|---|
| Linux (Steam Deck Holo 3.4.6) | 2023-04-04 |
| Linux (Ubuntu 18.04 aarch64) | 2023-04-13 |
| MacOS 10.13 (x64) | 2023-04-04 |
| MacOS 13.0 (arm64) | 2023-04-13 |
| Windows 10 (x64) + WSL Ubuntu | 2023-01-14 |
| Windows 11 (x64) + WSL Ubuntu | 2023-04-04 |
Install dependencies
OS-Specific Setup
- Windows WSL
- MacOS
- Linux
A) Ensure WSL ("WSL 2" in Windows 10) and the Ubuntu distribution are installed.
B) Install mercurial and subversion.
# Install support programs for the build and test commands
sudo add-apt-repository ppa:mercurial-ppa/releases
sudo apt-get update
sudo apt-get install mercurial subversion
sudo add-apt-repository --remove ppa:mercurial-ppa/releases
C) Install NodeJS
# Install bootstrap NodeJS and NPM within the WSL
curl -fsSL https://deb.nodesource.com/setup_16.x | sudo -E bash -
sudo apt-get install -y nodejs
Exit the WSL window and open a new one before proceeding:
# Switch to `n`-managed NodeJS
sudo npm i -g n
sudo n 16
D) Build and install a version of Git with proper SSL support:
# Git does not support OpenSSL out of the box, must do this
curl -LO https://github.com/paul-nelson-baker/git-openssl-shellscript/raw/main/compile-git-with-openssl.sh
chmod +x compile-git-with-openssl.sh
./compile-git-with-openssl.sh
E) Set git config core.autocrlf setting to false. The following commands
should be run twice, once within PowerShell and once within WSL bash:
git config --global --add core.autocrlf false
git config --global --unset core.autocrlf true
A) Run git. If Xcode or the command-line tools are not installed, you will be
asked to install. Click "Install" and run through the steps.
B) Open a terminal window and install Homebrew:
/bin/bash -c "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/HEAD/install.sh)"
C) Close the window, open a new terminal window, and disable analytics:
brew analytics off
To confirm analytics are disabled, run
brew analytics state
It should print Analytics are disabled.
D) Install Mercurial and Subversion:
brew install mercurial subversion
E) Install NodeJS
The official NodeJS site provides installers for "LTS" and "Current" releases. The "LTS" version should be installed.
Older versions of macOS are not compatible with newer versions of NodeJS. In
local testing, macOS 10.13 required NodeJS version 12.22.12
A) Install curl, mercurial, git, and subversion using the system package
manager. On Debian and Ubuntu systems, apt-get installs packages:
sudo apt update
sudo apt-get install curl git mercurial subversion
Other Linux distributions may use other package managers.
Steam Deck (click to show)
Desktop Mode on the Steam Deck uses pacman. It also requires a few steps.
0) Switch to Desktop mode and open Konsole
1) Set a password for the user by running passwd and following instructions.
2) Disable read-only mode:
sudo steamos-readonly disable
(When prompted, enter the password assigned in step 1)
3) Configure keyring:
sudo sh -c 'echo "keyserver hkps://keyserver.ubuntu.com" >> /etc/pacman.d/gnupg/gpg.conf'
sudo pacman-key --init
sudo pacman-key --populate
sudo pacman-key --refresh-keys
4) Install dependencies:
sudo pacman -S base-devel mercurial subversion
After installing mercurial and subversion, install NodeJS.
The official NodeJS site provides installers for "LTS" and "Current" releases. The "LTS" version should be installed.
After installing, if running node in the terminal fails with a glibc error,
an older version of NodeJS should be installed. For example, Ubuntu 18.04 does
not support Node 18 support Node 16.20.0.
Build from source tree
0) Clone the project:
git clone https://git.sheetjs.com/sheetjs/sheetjs
cd sheetjs
1) Install NodeJS modules for building the scripts
# Install dev dependencies
npm i
# Install global dependencies
sudo npm i -g [email protected] voc @sheetjs/uglify-js
Some of the dependencies are wildly out of date. While SheetJS aims to run in older versions of NodeJS and browsers, some libraries have chosen to break backwards compatibility. The specific versions are used because they are known to work and known to produce consistent results.
2) Initialize the test files:
make init
This step may take a while as it will be downloading a number of test files.
3) Run a short test, then run a build
# Short test
make test_misc
# Full Build
cd modules; make clean; make; cd ..
make dist
# Reset repo
git checkout -- .
Reproduce official builds
4) Run git log and search for the commit that matches a particular release
version. For example, version 0.19.3 can be found with:
git log | grep -B4 "version bump 0.19.3"
The output should look like:
$ git log | grep -B4 "version bump 0.19.3"
commit 333e4e40f9c5603bd22a811f54c61c20bc9e17ab <-- this is the commit hash
Author: SheetJS <[email protected]>
Date: Mon Apr 17 23:39:28 2023 -0400
version bump 0.19.3
5) Switch to that commit:
git checkout 333e4e40f9c5603bd22a811f54c61c20bc9e17ab
6) Run the full build sequence
make clean
cd modules; make clean; make; cd ..
make
make dist
7) To verify that the files are intact, use md5sum (md5 on MacOS).
The local checksum for the browser script can be computed with:
$ md5sum dist/xlsx.full.min.js
f5c73b5ddc4b431c909d11c2e1d7a8e0 dist/xlsx.full.min.js
The checksum for the CDN version can be computed with:
$ curl -L https://cdn.sheetjs.com/xlsx-0.19.3/package/dist/xlsx.full.min.js | md5sum -
f5c73b5ddc4b431c909d11c2e1d7a8e0 -
The two hashes should match.
8) To return to the HEAD commit, run
git checkout master
Test in web browsers
9) Start local server:
make ctestserv
The terminal will display a port number. For example:
Serving HTTP on 0.0.0.0 port 8000 (http://0.0.0.0:8000/)
10) Open a browser window and access http://localhost:8000, replacing 8000
with the port number from the terminal window.
Development
The xlsx.js and xlsx.mjs files are constructed from the files in the bits
subfolder. The build script (run make) will concatenate the individual bits
to produce the scripts.
To produce the dist files, run make dist. The dist files are updated in each
version release and should not be committed between versions.
Tests
The test_misc target runs the targeted feature tests. It should take 5-10
seconds to perform feature tests without testing against the full test battery.
New features should be accompanied with tests for the relevant file formats.
For tests involving the read side, an appropriate feature test would involve reading an existing file and checking the resulting workbook object. If a parameter is involved, files should be read with different values to verify that the feature is working as expected.
For tests involving a new write feature which can already be parsed, appropriate feature tests would involve writing a workbook with the feature and then opening and verifying that the feature is preserved.
For tests involving a new write feature without an existing read ability, please
add a feature test to the kitchen sink tests/write.js.