What is Stashing?
Stashing is the process by which a large dataset is broken into smaller chunks for uploading to Glide. Each chunk is uploaded to Glide independently (either sequentially or in parallel) to form the complete dataset. Once all data has been uploaded to the stash, the stash can then be referenced in other API calls to refer to the full dataset. This eliminates the need to include the entire dataset in the request itself, which may not be feasible due to its size.When to Use Stashing
You should use stashing when both of the following conditions are met:- You have a large dataset that you want to upload to Glide. Anything larger than 15MB should be broken up into smaller chunks and stashed.
- You want to perform an atomic operation using a large dataset. For example, you may want to perform an import of data into an existing table but don’t want users to see the intermediate state of the import or incremental updates while they’re using their application.
Stash IDs and Serials
The main components of a stash are its ID and the individual chunked data subsets, which are identified by serials. Both the ID and serial are values you define. The stash ID is a unique identifier for the stash that you define. You might use information that’s relevant to your domain, such as20240215-job32
or 2024-07-05T15-17-50Z_customer93ak
, a UUID, or any other unique identifier.
Each chunk of data that is uploaded to the stash is identified by a unique serial. The sort order of the serials indicates the order of the chunks in the overall datset. If all serials can be parsed as integers, numerical sort order is used, otherwise sorting is done lexicographically according to each character’s Unicode code point value. If the order of data chunks in the stash is important, using integers as serials (e.g., 1
, 2
, etc…) is recommended.
The maximum length for both stash IDs and serials is 256 characters. They may only contain letters, numbers, hyphens, and underscores, and must start with a letter or number.