Data Storage on KaiOS

KaiOS offers standard Web Storage APIs (localStorage and sessionStorage), indexedDB, document.cookie, and the Cache API as well as platform-specific APIs like Device Storage and Data Store. The ideal API for your use case depends on factors including data size, security, and simplicity. Keep in mind that KaiOS devices have limited storage available (i.e. the Blu Zoey Smart only has 512MB), and many users will not have an external micro SD card, so its best to store the minimum data necessary and assume most persistence is best-effort.

Web Storage (localStorage and sessionStorage)

localStorage is by far the simplest Web Storage API. It offers a synchronous key-value data store with one data type–strings–capped at 5MiB per application (configured via the dom.storage.default_quota preference). Since JSON can serialize nearly any object to a string, it’s possible to store more complex data as well. Ideally, localStorage is used for small amounts of persisted, non-sensitive data like the timestamp when an API was last polled or user style and language preferences.

⚠️ Warning: wrap localStorage calls to catch errors like NS_ERROR_DOM_QUOTA_REACHED. Beware that certain user preference changes may act similarly to private browsing, effectively setting the storage limit to 0.

Unfortunately, localStorage is not co-operatively scheduled and because it is fully synchronous, it blocks all execution until completion. As a result, it’s best not to retrieve data from localStorage in loops. It’s also only available from the main thread, so it’s inaccessible from Web Worker or Service Workers.

• 👍 Easy to use
• 👍 Standard API
• 👎 5MiB limit
• 👎 Blocks main thread
• 👎 Not accessible via Workers
• 👎 String serialization

ℹ️ Pro Tip: sessionStorage works just like localStorage, but only persists while your app is open. sessionStorage a great way to cache up to 5MiB of data that you might otherwise have stored in memory.

IndexedDB

IndexedDB is a standard Web API that offers a NoSQL name-value database with indexes, transactions, fast asynchronous lookups, and the ability to store objects and binary blobs without string serialization. However, it gets a bad reputation because its APIs are complex and use outdated callback syntax instead of modern approaches like Promise or async and await.

• 👍 Asynchronous
• 👍 Standard API
• 👍 Binary Data
• 👍 Main Thread & Workers
• 👎 Complex API
• 👎 Unintelligible Errors
• 👎 No Storage Utilization Information

Unlike localStorage, IndexedDB is available in all contexts (main thread, web workers, and service workers). Further, it’s main shortcoming can be mitigated by using a library like idb that improves usability by wrapping API calls with Promise and providing TypeScript type definitions.

⚠️ Warning: overuse of IndexedDB can cause device-wide performance degradation and frequent errors that may necessitate a factory reset. Use caution when storing large amounts of data in IndexedDB.

Cookies

Cookies are the technology that allows for persisted session state, enabling users to log in and stay logged in for a fixed period of time. However, they’re also the technology that allows advertisers to track users across websites. The document.cookie is a simple API for reading and writing cookies.

Apps can store up to 20 named cookies with a maximum string length of 4Kb (80kb total). All data is serialized to URL-safe strings (no commas, semicolons, whitespace, etc), and sent along with HTTPS requests for the appropriate domains via the Cookie header.

• 👍 Simple
• 👍 Standard API
• 👍 Essential for logins
• 👎 80kb limit
• 👎 String serialization
• 👎 May require consent warning (i.e. per GDPR/ ePD)

Cookies are the only standard data storage technology that can persist beyond when an application is uninstalled (but not a factory reset). This means that Cookies can be a powerful tracking mechanism for deduplicating users who might uninstall and reinstall your application.

⚠️ Warning: app origins changed between KaiOS 2.5 (i.e. app://podlp.com) and KaiOS 3.0 (i.e. https://podlp.localhost/). As a result, cookies in KaiOS 3.0 cannot be set client-side for remote origins like https://podlp.com, and cookies set using the Set-Cookie header are not accessible via the document.cookie property.

Cache API

The Cache API offers browser-dependent persistence for Request and Response object pairs. It’s primarily used in Service Workers to cache responses for faster subsequent retrieval. Persistence is best effort and will be preferentially evicted when there is storage pressure. As a result, the Cache API should be treated as ephemeral.

• 👍 Asynchronous, Promise-based
• 👍 Standard API
• 👍 Offline Usage
• 👎 Ephemeral
• 👎 Request/ Response Only

⚠️ Warning: watch out when caching Range requests typical for streaming audio and video.

Device Storage

The Device Storage API provides access to the file system within an app. It’s only available to Privileged apps, and requires a different permission for each storage name.

• apps - Certified only, used to store user data needed by apps
• music - Location for music and sounds
• pictures - Location for pictures and wallpapers
• sdcard - General data storage location; may be internal (emulated sdcard)
• videos - Location for videos and movies

Device Storage transparently maps storage names to mount points that are consistent across all applications. This means that files written by one application (i.e. the Camera app) are accessible by others apps (i.e. the Gallery app), with the appropriate permission. Permissions are declared in manifest.webapp and require the access property, which can be readonly or readwrite, depending on whether write access is required.

1"permissions": {
2  "device-storage:music": { "access": "readonly" },
3  "device-storage:pictures": { "access": "readwrite" }
4}

Device Storage accepts Blobs which are written to files. Here’s a simple example writing a text file to the sdcard storage name.

 1let sdcard = navigator.getDeviceStorage("sdcard");
 2let file = new Blob(["Test text file"], { type: "text/plain" });
 3
 4let request = sdcard.addNamed(file, "text-file.txt");
 5
 6request.onsuccess = (e) => {
 7  console.log('File "' + e.target.result.name + '" successfully wrote on the sdcard storage area');
 8};
 9
10request.onerror = (e) => {
11  console.warn('Unable to write the file: ' + e.target.error);
12};

Device Storage APIs to read and write files are asynchronous and return DOMRequest objects that provide onsuccess and onerror event handlers. Alternatively, DOMRequest is “thennable” and can return a Promise.

⚠️ Warning: Don’t be fooled! Some APIs like getEditable were never fully implemented, returning a File instead of a FileHandle. Don’t rely solely on documentaion, always validate on commercial devices.

The full API signature is available at DeviceStorage.webidl, but here’s a summary of key functions on the DeviceStorage object returns by navigator.getDeviceStorage.

• addNamed - Create a new file
• appendNamed - Append to an existing file
• get - Retrieve an existing file
• delete - Delete an existing file
• enumerate - Enumerate files in a directory
• freeSpace - Amount of free storage space available
• usedSpace - Amount of storage space in use
• isDiskFull - True if free space of /data is under 30MB (preference: disk_space_watcher.low_threshold)

ℹ️ FYI: the default KaiOS Music and Video apps do not honor .nomedia hidden files, so all audio or video files with specific extensions (i.e. .mp3) will be displayed in these pre-installed apps.

It is worth pointing out that unlike other data storage technologies, Device Storage allows appending to files. This means that data can be streamed in chunks (i.e. audio and video) and persisted to a single location, without bumping up against memory limits. It’s also the only KaiOS Storage API that persists beyond uninstalls and provides information on used and available space, allowing apps to responsibly manage data stored while providing users with warnings regarding utilization.

Finally, the Device Storage API is the only API that requires user-granted permission prompted for privileged apps. Calling navigator.getDeviceStorage will prompt the user to grant permission one-time, with an option to remember their choice. Permission can be revoked at any time from the Settings app, in which case API calls will trigger a SecurityError.

• 👍 Asynchronous
• 👍 File storage
• 👍 Append API
• 👎 Main thread only
• 👎 Always shared
• 👎 Cannot restrict to specific origins

Data Store

Data Store is a KaiOS-specific storage mechanism to store and share data. The API acts as an intermediary store to allow multiple applications to share data between one another. Behind the scenes, Data Store is actually a wrapper around IndexedDB where data is stored in various revisions under the database name, DataStoreDB. KaiOS manages read and write access to data stores based on declarations in manifest.webapp.

 1"datastores-owned": {
 2  "myData": {
 3    "access": "readwrite",
 4    "description": "my data store"
 5  }
 6},
 7"datastores-access": {
 8  "myData": {
 9    "access": "readonly",
10    "description": "Read and modify my data store"
11  }
12}

Privileged apps can declare Data Stores that they create using datastores-owned, and Data Stores that they access using datastores-access. Similar to Device Storage, the access property is required to allow readonly or readwrite access.

• 👍 Asynchronous
• 👍 Shared storage
• 👍 Main Thread & Workers
• 👎 Cannot restrict to specific origins
• 👎 Poorly documented

KaiOS Data Storage Summary

Each storage technology writes data to a different location. Here are some default locations. /<profile>/ denotes the profile location, /data/b2g/mozilla/*.default, and * denotes a random identifier for Mozilla’s default profile where bookmarks, passwords and other user data are stored.

Recap

In addition to the APIs mentioned above, the web is littered with a graveyard of deprecated vendor-specific storage solutions including window.name, WebSQL, and AppCache. Here’s a quick recap of how you might use each storage technology.

• A cookie (or two) for session state
• localStorage for persisted user preferences
• sessionStorage for larger, ephemeral state
• IndexedDB for purgeable binary data
• The Cache API for serving responses offline
• Device Storage for files (i.e. music and pictures)
• Data Store for shared data (i.e. contacts)

KaiOS provides many methods for storing and retrieving data client-side. Picking the right solution is important, since KaiOS has partially-implemented APIs, client-side data migrations can be challenging, and storage pressure warnings can force factory resets. If you need an experienced partner to inform your KaiOS data storage strategy, contact the author from the About page.

• Previous
  What's missing from KaiOS development?
• Next
  Practical KaiOS Optimization Tips