# MinusOneDB Documentation — Full Text Build, query, and manage MinusOneDB environments with the m1 client, Environment API, Ops API, guides, and release notes. Generated from docs-vp/*.md in sidebar order. CONTRIBUTING.md is intentionally excluded. Canonical docs URL: https://www.minusonedb.com/docs/ -------------------------------------------------------------------------------- > Source: https://www.minusonedb.com/docs/ # Introduction Welcome to MinusOneDB. Here you will find documentation, solutions and samples showing how to use the MinusOneDB platform. People interested in event capture should look at our [event capture solution](https://www.minusonedb.com/docs/event-capture). Alternatively, use our [ops api](https://www.minusonedb.com/docs/ops-api), [environment api](https://www.minusonedb.com/docs/environment-api), and [m1 client](https://www.minusonedb.com/docs/m1-client) documentation to to build out your own solutions. You can immediately create and play with your own [free trial](https://www.minusonedb.com/docs/free-trial) here. Most of the examples follow linux/unix conventions though we hope they will be fairly straightforward to follow in other environments. If, for this or any other reason, you run into any issues, need more help, or have a problem not covered by any of our existing solutions or samples, [please let us know](mailto:support@minusonedb.com). We are always here to help! -------------------------------------------------------------------------------- > Source: https://www.minusonedb.com/docs/m1-client # m1 client The m1 client is the command line interface to minusonedb. In addition to allowing you to execute all of the endpoints available in [ops](https://www.minusonedb.com/docs/ops-api) or one of your [environments](https://www.minusonedb.com/docs/environment-api), the m1 client simplifies [authentication](https://www.minusonedb.com/docs/m1-client#authenticating) and operations that require coordination between `https://ops.minusonedb.com` and your environments. For brevity's sake only m1 authentication and orchestration operations are described below; see the service api references for additional m1 sample usages. ### Getting Started _Step 1:_ Download the latest [m1 client](https://docs.minusonedb.com/assets/m1.zip) zip file. _Step 2:_ Unzip the `m1.zip` file to a location on your computer. _Step 3:_ Add the unzipped m1 directory to your path based on your operating system's instructions. _Step 4:_ The m1 client requires java. Install [java](https://www.minusonedb.com/docs/troubleshooting#java-setup) if it is not already present on your computer. _Step 5:_ Try running `m1` in a terminal. You should see something like: `Usage: m1 ` If the m1 command is not found, make sure the m1 directory containing the m1.jar zip was successfully added to your path. If java is not found, doublecheck your java installation steps. ### Signing Up Only an email address is required to obtain a MinusOneDB ops login. Your ops login will let you create a free trial or manage any accounts and environments that other MinusOneDB users have shared with you. We will not send outbound marketing emails to you; if you receive an email from us it is only in support of the service we provide; our [terms of use](https://www.minusonedb.com/terms) define how we want to do business with you. To sign up, run: `m1 signup` You will be prompted for a username (which must be a valid email address) and a password. If successful, those credentials will be saved to a local directory and used subsequently to get tokens to access ops services. From here you can keep on reading about how to use the m1 client or go [start your free trial](https://www.minusonedb.com/docs/free-trial). ### Authenticating > Connect to `https://ops.minusonedb.com` ``` m1 auth ``` > Connect to your environment at `https://envName-accountName.minusonedb.com` ``` m1 auth envName-accountName ``` Connect to `https://ops.minusonedb.com` by running the following: `m1 auth` You will be prompted for your MinusOneDB username (email) and password; these credentials will be saved to a local directory and used subsequently to get tokens to access ops services. **Note**: Do not use m1 auth to cache credentials on a shared computer; others may be able to access your MinusOneDB resources. Connect to one of your environments by running the following command, replacing envName and accountName appropriately: `m1 auth envName-accountName` As with ops, you will be prompted for your credentials to access your environment. You will not need to rerun `m1 auth` unless your credentials change. ### Creating an Environment > A sample m1 environment configuration file ``` { "environment": { "create" : { "servers": 1 }, "init": { "username": "admin", "password": "change this password" }, "stores": [{\ "store": "index",\ "type": "standard",\ "shards": 1,\ "replicas": 1\ }], "sessionStores": [{\ "store": "stash",\ "shards": 1,\ "replicas": 1\ }], "configure": [{\ "system": {\ "geo": "true",\ "publish-permissive": "true"\ }\ },{\ "user/update": {\ "username": "admin",\ "rights": ["admin","get","publish","schema","sessionUpdate","sessionQuery"]\ }\ },{\ "user/add": {\ "username": "flush",\ "password": "change this password",\ "rights": ["publish","sessionUpdate","sessionQuery"]\ },\ "user/add": {\ "username": "instrument",\ "password": "this password does not matter",\ "rights": ["publish", "sessionUpdate"]\ }\ },{\ "store/autocommit": {\ "store": "index",\ "seconds": 2\ }\ },{\ "schema/add": {\ "properties": "@../schema/instrument.json"\ }\ },{\ "schema/add": {\ "properties": "@../schema/geo.json"\ }\ },{\ "schema/add": {\ "properties": "@../schema/sample.json"\ }\ }] } } ``` Create a new environment by running the following: `m1 orchestrate deploy envName environment-configuration.json` `envName` is the name of the new environment to create. The [configuration file](https://www.minusonedb.com/docs/m1-client#configuration-file-format) specifies desired environment parameters -- the number of data processing servers, data store and session store topology, users and rights, as well system property configuration. Sample environment template files are available with our downloadable samples. To use an environment template file: _Step 1:_ Make a copy of the environment template file. _Step 2:_ Set your admin username and password in the `init` block. _Step 3:_ Create any additional users who should have access this environment. The example configuration below creates users `flush` and `instrument`. _Step 4:_ You can make any other desired edits to the template file here. _Step 5:_ Run `m1 orchestrate deploy envName your-copied-config-from-step-1.json`. **Note**: Deploying a new environment will typically take 5-10 minutes. ### Adding Stores > Add a data store named index and a session store named stash ``` m1 orchestrate addStores test-m1 addStores.json ``` > An example addStores.json that adds a store and a session store: ``` { "environment": { "stores": [{\ "store": "index",\ "type": "standard"\ }], "sessionStores": [{\ "store": "stash"\ }] } } ``` Create and add any number of data stores or session stores to an existing environment by running the following: `m1 orchestrate addStores envName-accountName addStores.json` As a convenience, you can use an environment config file as an argument to `m1 orchestrate addStores`; elements other than the `stores` and `sessionStores` entries will be ignored. `orchestrate addStores` adds [data store](https://www.minusonedb.com/docs/env-data-stores#store-add) and [session store](https://www.minusonedb.com/docs/env-session-stores#session-add) configurations to an environment and then provisions corresponding [data store](https://www.minusonedb.com/docs/env-data-stores#env-store-create) and [session store](https://www.minusonedb.com/docs/env-session-stores#env-session-create) server resources via `https://ops.minusonedb.com`. ### Dropping a Store > Drop a store named index ``` m1 orchestrate dropStore store test-m1 index ``` > Drop a session store named stash ``` m1 orchestrate dropStore session test-m1 stash ``` Drop and destroy a data store by running the following: `m1 orchestrate dropStore store envName-accountName dataStoreToDrop` This operation [drops](https://www.minusonedb.com/docs/env-data-stores#store-drop) the data store configuration in the environment and then [destroys](https://www.minusonedb.com/docs/ops-api#env-store-destroy) the data store server resources via `https://ops.minusonedb.com`. Drop and destroy a session store by running the following. `m1 orchestrate dropStore session envName-accountName sessionStoreToDrop` This operation [drops](https://www.minusonedb.com/docs/env-session-stores#session-drop) the session store configuration in the environment and then [destroys](https://www.minusonedb.com/docs/ops-api#env-session-destroy) the session store server resources via `https://ops.minusonedb.com`. ### Renaming a Store > Rename a store named index to index2 ``` m1 orchestrate renameStore store test-m1 index index2 ``` > Rename a session store named stash to stash2 ``` m1 orchestrate renameStore session test-m1 stash stash2 ``` Rename a data store by running the following: `m1 orchestrate renameStore store envName-accountName dataStoreToRename` This operation [renames](https://www.minusonedb.com/docs/env-data-stores#store-rename) the data store configuration in the environment and then [renames](https://www.minusonedb.com/docs/ops-api#env-store-rename) the data store server resources via `https://ops.minusonedb.com`. Rename a session store by running the following. `m1 orchestrate renameStore session envName-accountName sessionStoreToRename` This operation [renames](https://www.minusonedb.com/docs/env-session-stores#session-rename) the session store configuration in the environment and then [renames](https://www.minusonedb.com/docs/ops-api#env-session-rename) the session store server resources via `https://ops.minusonedb.com`. ### Configuration File Format > eventsSessionStore.json template ``` { "environment": { "create" : { "servers": 1 }, "init": { "username": "admin", "password": "change this password" }, "stores": [{\ "store": "index",\ "type": "standard",\ "shards": 1,\ "replicas": 1\ }], "sessionStores": [{\ "store": "stash",\ "shards": 1,\ "replicas": 1\ }], "configure": [{\ "system": {\ "geo": "true",\ "publish-permissive": "true"\ }\ },{\ "user/update": {\ "username": "admin",\ "rights": ["admin","get","publish","schema","sessionUpdate","sessionQuery"]\ }\ },{\ "user/add": {\ "username": "flush",\ "password": "change this password",\ "rights": ["publish","sessionUpdate","sessionQuery"]\ },\ "user/add": {\ "username": "instrument",\ "password": "this password does not matter",\ "rights": ["publish", "sessionUpdate"]\ }\ },{\ "store/autocommit": {\ "store": "index",\ "seconds": 2\ }\ },{\ "schema/add": {\ "properties": "@../schema/instrument.json"\ }\ },{\ "schema/add": {\ "properties": "@../schema/geo.json"\ }\ },{\ "schema/add": {\ "properties": "@../schema/sample.json"\ }\ }] } } ``` The environment configuration file is a json map describing an environment. A fairly complete example configuration is shown above; a description of each key element of the configuration file is described below: \- environment \- create: Parameters for [env/create](https://www.minusonedb.com/docs/ops-api#env-create) (like the number of data processing servers). \- init: Your initial admin user/password. Note how this element corresponds to [init](https://www.minusonedb.com/docs/env-operations#init). \- stores: A list of data stores to [add](#adding-stores). \- sessionStores: a list of session stores to [add](#adding-stores). \- configure: a list of maps representing environment service operations to configure a store. Note how a [user/update](https://www.minusonedb.com/docs/environment-api#user-update), [/system](https://www.minusonedb.com/docs/env-system-config#system), or [schema/add](https://www.minusonedb.com/docs/environment-api#schema-add) elements in the example configuration relate to the corresponding api method. This pattern can be extended to other [environment](https://www.minusonedb.com/docs/environment-api) service endpoints. -------------------------------------------------------------------------------- > Source: https://www.minusonedb.com/docs/free-trial # Start your free trial! The best way to learn how to use new technology is to play with it. To that end, we offer a 1 week free trial that allows you to use minusonedb and judge its value for yourself. Please [contact us](mailto:support@minusonedb.com) if there is anything that we can do to make your trial more effective. We are always here to help. To set up a free trial environment: _Step 1:_ Set up the [m1 client](https://www.minusonedb.com/docs/m1-client) and [sign up](https://www.minusonedb.com/docs/m1-client#signing-up) if you have not already done so. _Step 2:_ Create a trial environment by running the following, changing the parameters as appropriate: `m1 orchestrate deploy yourtrialname-trial events.json` Replace `yourtrialname` with whatever you would prefer (the string can only contain alphanumeric characters and cannot be in use by another trial user). Replace `events.json` with an m1 environment [config file](https://www.minusonedb.com/docs/m1-client#configuration-file-format) that suits your needs. The various [event capture](https://www.minusonedb.com/docs/event-capture) sample walkthroughs contain example templates or you can make your own. The `orchestrate` command will take a 5-10 minutes to complete. Once your trial environment is up, you can interact with it in the same way you would a paid minusonedb environment: [see](https://www.minusonedb.com/docs/ops-environments#env-get) your trial environment configuration, [register](https://www.minusonedb.com/docs/ops-environments#env-bucket-register) buckets to load your own data from S3, or use any of the [environment services](https://www.minusonedb.com/docs/environment-api) to interact with your environment now hosted at `https://yourtrialname-trial.minusonedb.com`. Note the following limitations with trial environments: - Trial environments can have at most 1 data processing server. - Trial environments can have at most 1 indexed store with 1 shard. - You can only have 1 trial environment running at a time. - You are allowed 1 week of trial usage across all of your trial environments. If at any point you wish to pause or end your trial, simply [destroy](https://www.minusonedb.com/docs/ops-environments#env-destroy) your active trial environment (you can use any time you still have left on a future free trial). If you need to extend your trial or you want to become a minusonedb customer, [reach out to us](mailto:support@minusonedb.com). -------------------------------------------------------------------------------- > Source: https://www.minusonedb.com/docs/event-capture # Event capture Our event capture solution should have everything you need to understand user behavior in any context. The samples below illustrate common patterns for identifying, storing, and querying events. You can create events when a customer adds an item to a shopping cart, swipes right on a mobile app, leaves your site or anything else of interest to you. Or you can apply the same patterns to capture data from devices for monitoring or telemetry purposes. To get started by instrumenting a website or other JavaScript app continue to the next section. If you would prefer to explore using simulated traffic jump to the [Event Simulator](https://www.minusonedb.com/docs/event-capture#event-simulator) setup instructions. This can be useful for testing before your website goes live or for testing large volumes of events. ## Instrument Your Website See how users interact with your web application. **_Setup Time:_** 20 minutes **_Run Time:_** Immediate This tutorial shows how to instrument your web application using MinusOneDB. This will allow you to understand how users are interacting with your website in great detail. Once you have completed the steps below, user interaction events will be sent from your website to your minuseondb environment. These events can be queried in near real-time to understand how users are interacting with your site. This tutorial will also guide you through some common customization paths to further enrich the data you can capture. ### Prerequisites _Step 1:_ Download and unzip the [instrumentation](https://docs.minusonedb.com/assets/instrument.zip) sample. _Step 2:_ Set up the [m1 client](https://www.minusonedb.com/docs/m1-client) if you have not already done so. _Step 3:_ You will need a MinusOneDB environment: - If you have never used MinusOneDB before, you can set up a [free trial](https://www.minusonedb.com/docs/free-trial) environment and then [create](https://www.minusonedb.com/docs/m1-client#creating-an-environment) an environment using the `events.json` environment template file available in your download. - If you have a MinusOneDB account but have not set up an environment yet, [create](https://www.minusonedb.com/docs/m1-client#creating-an-environment) an environment using the `events.json` environment template file available in your download. - If you already have your target environment follow the instructions below to instrument your website and begin capturing data. ### Instructions _Step 1:_ Copy the `instrumentation.js`, `M1Events.js`, and `lib/jquery-3.6.0.min.js` files from the instrument/src sample directory into the same folder as your web application's other JavaScript files. There is no need to copy the jQuery file if your site already uses it. > To make a new user with appropriately restrictive rights: ``` m1 envName-accountName user/add -username instrument -password ChangeMeNow! -rights '["publish"]' ``` ``` curl https://envName-accountName.minusonedb.com/user/add \ -d 'username=instrument&password=ChangeMeNow!&rights=["publish"]' \ -H "m1-auth-token: $adminToken" ``` > To restrict the rights of an existing user to only publish: ``` m1 envName-accountName user/update -username instrument -rights '["publish"]' ``` ``` curl https://envName-accountName.minusonedb.com/user/update \ -d 'username=instrument&rights=["publish"]' \ -H "m1-auth-token: $adminToken" ``` _Step 2:_ Set the environment name and user account in the "REQUIRED PARAMETERS" section of the copied `instrumentation.js` file. Typically this account should be a system account not used for any other purpose. This account should only have the publish right. _Step 3:_ Source the JavaScript files copied above by adding the following lines to the head of your html file (or other location where you import JavaScript files); this is often in an index.html file. `` `` `` **Note**: You may need replace the js/ directory in these examples with the directory from which your application sources JavaScript files. It can be omitted if the files are added to the top level directory. > Add localhost.somethingunique to the cors domain whitelist: ``` m1 envName-accountName system/cors/add -domains localhost.somethingunique ``` ``` curl https://envName-accountName.minusonedb.com/system/cors/add \ -d "domains=localhost.somethingunique" \ -H "m1-auth-token: $myToken" ``` _Step 4:_ The sample `instrumentation.js` file contains reasonable defaults for instrumenting a single page web app; `sessionStart` events are recorded when a user first comes to a site and `view` events are created as users navigate via hyperlinks. `instrumentation.js` contains some other sample event capture patterns that you may want to adopt or customize for your own needs. _Step 5:_ You will need to add your website's domain to your environment's CORS domain whitelist. You can add as many domains as needed. If you are testing on localhost, add a unique entry to your [hosts](https://www.howtogeek.com/27350/beginner-geek-how-to-edit-your-hosts-file/) file that resolves to localhost, i.e., `127.0.0.1 localhost.somethingunique` and then add that as whitelist entry using one of the commands below. We encourage you to add the entry to hosts rather than directly adding localhost as a whitelisted domain to prevent any chance of running into a CORS related vulnerability. > See the 5 most recent events: ``` m1 envName-accountName query -store index -q '*' -sort "_m1.receivedUTC desc" -rows 5 ``` ``` curl https://envName-accountName.minusonedb.com/query \ -d "store=index&q=*&sort=_m1.receivedUTC desc&rows=5" \ -H "m1-auth-token: $myToken" ``` > See events by state: ``` m1 envName-accountName query -store index -q '*' -json '{ "facet": { "by_state": { "field": "_m1.geo.subdivision_1_iso_code", "type": "terms" } } }' ``` ``` curl https://envName-accountName.minusonedb.com/query -d 'store=index&q=*&json={ "facet": { "by_state": { "field": "_m1.geo.subdivision_1_iso_code", "type": "terms" } } }' -H "m1-auth-token: $myToken" ``` _Step 6:_ Open up your site in a browser; if your site was hosted at localhost, be sure to use the hosts entry you created above in place of localhost (i.e. `http://localhost.somethingunique:8080/index.html` rather than `http://localhost:8080/index.html`). As you navigate around your site, events will flow into your MinusOneDB environment; you can query your events using our [reporting](https://reports.minusonedb.com/) application or by using the sample queries below. **Troubleshooting Tip:** If the queries above are not returning results, open your browser's console and check if there are any errors. Doublecheck each of the steps above, making sure your site has loaded the custom JavaScript, your user credentials are correct and there are no other obvious errors. If you get stuck, reach out to us. Congratulations! You have now successfully instrumented your website. You are capturing web events as they occur and you can query this data in near real-time. Continue reading for some guidance on taking this to production. ### Next Steps Before deploying these changes to your production web site: > Add minusonedb.com to the cors domain whitelist: ``` m1 envName-accountName system/cors/add -domains minusonedb.com ``` ``` curl https://envName-accountName.minusonedb.com/system/cors/add \ -d "domains=minusonedb.com" \ -H "m1-auth-token: $myToken" ``` _Step 1:_ In `instrumentation.js`, set the async parameter to true. This allows the MinusOneDB servers to optimize the storage of events in your production environment. _Step 2:_ Add your production website domain to your MinusOneDB environment's cors whitelist. Replace minusonedb.com with your domain as in the examples below. _Step 3:_ There are a number of sections in instrumentation.js that may be of further interest. In particular, consider adding [heartbeat](#heartbeats) events to gain a much richer time series based perspective of your users' behavior. ## Heartbeats See how users interact with your web application **over time**. **_Setup Time:_** 5 or 25 minutes (depends on whether you have worked through the [instrumentation sample](#instrument-your-website)). **_Run Time:_** Immediate Heartbeats are periodic events that record that a user is still present. They allow you to understand how long a user stays on your site and when they leave it. Heartbeats directly allow you to calculate the number of concurrent users on your site that can then be broken down by geography or individual page. In addition to the notion of presence, any attributes available on heartbeat events allow for a time series perspective that is not possible otherwise. What pages on your site do users spend the longest time on? What is the most frequently viewed viewed page right before a user leaves your site? Heartbeat events can help answer these and many other questions. The volume of heartbeats events will typically dwarf that of user interaction events. While we have found a great deal of benefit in heartbeat events at a 5 second frequency, some customers will want to optimize for cost. This can done by lowering the heartbeat frequency or by not persisting heartbeats at all and instead using them only to calculate user departure. See [configuring heartbeat events with a session store](#heartbeats-on-a-budget) or continue reading to persist all heartbeat events. We strongly encourage persisting heartbeats to start. The persisted heartbeats will be more cost-effective until you have collected 1-2 million hours of recorded user interactions. By then you will have a better understanding of the value of these and other events to your business. ### Prerequisites _Step 1:_ Work through the steps of the [Instrument Your Website](#instrument-your-website) sample above. ### Instructions > See the 3 most recent heartbeat events: ``` m1 envName-accountName query -store index -q "event.type:heartbeat" -sort "_m1.receivedUTC desc" -rows 3 ``` ``` curl https://envName-accountName.minusonedb.com/query \ -d "store=index&q=event.type:heartbeat&sort=_m1.receivedUTC desc&rows=3" \ -H "m1-auth-token: $myToken" ``` _Step 1:_ In the `instrumentation.js` file you edited above, configure an appropriate HEARTBEAT\_INTERVAL\_MS value and uncomment the line that saves all heartbeats. _Step 2:_ Open up your site in a web browser. Heartbeat events will periodically be sent to your minusonedb environment. You can query your events using our [reporting](https://reports.minusonedb.com/) application or by using the sample queries below. ## Heartbeats on a Budget Capture heartbeats in ephemeral storage to understand concurrent user metrics while reducing costs for large sites. **_Setup Time:_** 10 or 30 minutes (depends on whether you have worked through the [instrumentation sample](#instrument-your-website)). **_Run Time:_** Immediate While heartbeat events offer many benefits, their number will typically exceed the number of non-heartbeat user interaction events. You may not wish to persist a high volume of potentially similar heartbeat events, while still desiring analytics around user presence. [Rather](#heartbeats) than storing heartbeats in a datalake and indexed store, this sample shows how a client can send heartbeats to a session store and have a separate session flusher process determine when sessions have timed out. In other words, this sample allows you to record only the **last** (and most analytically interesting) heartbeat for a given session and foregoes persisting the intervening ones. Note that the added cost of the session store is roughly the equivalent in price of storing an additional 1-2 million hours of user interactions; at smaller volumes persisting heartbeats will be more cost-effective. ### Prerequisites _Step 1:_ Work through the steps of the [Instrument Your Website](#instrument-your-website) sample. You can save a step if you use the `eventsSessionStore.json` environment template file rather than the `events.json` described in that sample. \*Step 2: Download and unzip the [session flusher](https://docs.minusonedb.com/assets/flusher.zip) sample. > An example addSessionStore.json to add a new session store with name `stash`: ``` { "environment": { "stores": [\ ], "sessionStores": [{\ "store": "stash",\ "shards": 1,\ "replicas": 1\ }] } } ``` If you have already created an environment from `events.json` you can modify it to look like the `eventsSessionStore` environment by performing the following operations. These commands will add a session store, create a `flush` user with appropriate permissions, and allow the `instrument` user to write to the session store. `m1 orchestrate addStores envName-accountName envKey addSessionStore.json` `m1 envName-accountName user/add -username flush -password "change this password" -rights '["publish", "sessionUpdate", "sessionQuery"]'` `m1 envName-accountName user/update -username instrument -rights '["publish", "sessionUpdate"]'` _Step 3:_ Install and configure [python](https://www.minusonedb.com/docs/troubleshooting#python-setup) if you have not already done so. ### Instructions **Configure the Session Flusher** > Sample config.json.sample for reference: ``` { "server" : "server name (ex: https://live-m1.minusonedb.com)", "session" : "session name (ex: stash)", "username" : "username", "password" : "password", "timeoutSeconds" : "time (in seconds) before the flusher considers a session to have timed out; you'll generally want this to be 3-5x your heartbeat period." } ``` _Step 1:_ In the `flusher/src` directory, copy `config.json.sample` to `config.json`. Edit your environment, environment credentials, session store, and data store in `config.json` _Step 2:_ Run `python3 -u flusher.py config.json` _Step 3:_ You should see some output saying that there is "nothing to flush". `flusher.py` checks every few seconds to see if there are timed out sessions. _Step 4:_ Leave `flusher.py` running; you will come back to it after configuring `instrumentation.js` **Configure `instrumentation.js`** _Step 1:_ Open the `instrumentation.js` file you have already edited and find the "Using a session store with heartbeats" section. > See heartbeats in the session store: ``` m1 envName-accountName session/query -store stash -q "*" ``` ``` curl https://envName-accountName.minusonedb.com/session/query \ -d "store=stash&q=*" \ -H "m1-auth-token: $myToken" ``` _Step 2:_ Configure the session store name, configure the heartbeat frequency, and uncomment blocks of code as directed in `instrumentation.js` _Step 3:_ Open up your site in a web browser. Heartbeat events will be periodically sent to your configured session store. _Step 4:_ Query for heartbeats in the session store. You should see a document representing your active session. - **Troubleshooting Tip:** If your session store query returns no results, make sure there are no JavaScript errors and doublecheck the configuration steps above. > See persisted sessionTimeout events: ``` m1 envName-accountName query -store index -q "type:sessionTimeout" -rows 5 ``` ``` curl https://envName-accountName.minusonedb.com/query \ -d "store=index&q=type:sessionTimeout" \ -H "m1-auth-token: $myToken" ``` _Step 5:_ Now close the browser that was visiting your site -- heartbeats will no longer be sent to the session store. Navigate back to where you ran flusher.py. After the appropriate amount of time has elapsed (configurable via the `timeoutSeconds` parameter in the flusher.py config file), you will see a "Preparing to flush" message in the flusher.py output signifying that `flusher.py` found your session that has timed out. _Step 6:_ Additionally, you will see a `sessionTimeout` persisted event recording the time your session timing out. You can use the sample query below to retrieve these events. Congratulations! You now have system that allows you to determine the number of users that are actively using your site at any point in time at a minimal cost. ## Event Simulator Generate synthetic web events to familiarize yourself with the MinusOneDB event capture solution rather than [instrumenting](#instrument-your-website) your own website. **_Setup Time:_** 20 minutes **_Run Time:_** Immediate This python sample simulates user interactions on a web page. Use it to quickly create some test data to gain familiarity with MinusOneDB concepts. Run a number of simulator processes concurrently to simulate high traffic. Use in concert with the [session flusher](#heartbeats-on-a-budget) to test your own session flushing criteria. ### Prerequisites _Step 1:_ Download and unzip the [event simulator](https://docs.minusonedb.com/assets/simulator.zip) sample. _Step 2:_ Set up the [m1 client](https://www.minusonedb.com/docs/m1-client) if you have not already done so. _Step 3:_ You will need a MinusOneDB environment: - If you have never used MinusOneDB before, you can set up a [free trial](https://www.minusonedb.com/docs/free-trial) environment and then [create](https://www.minusonedb.com/docs/m1-client#creating-an-environment) an environment using the `events.json` environment template file available in your download. - If you have a MinusOneDB account but have not set up an environment yet, [create](https://www.minusonedb.com/docs/m1-client#creating-an-environment) one using the `eventsSessionStores.json` environment template file available in your download. - If you already have your target environment follow the instructions below to run the event simulator. _Step 4:_ Install and configure [python](https://www.minusonedb.com/docs/troubleshooting#python-setup) if you have not already done so. ### Instructions > config.json.sample for reference: ``` { "server" : "server name (ex: https://live-m1.minusonedb.com)", "session" : "session name (ex: stash). Set this param to null to simulate persisting heartbeats", "username" : "username", "password" : "password" } ``` In the simulator samples directory, copy `config.json.sample` to `config.json`. Edit your environment, environment credentials, session store, and data store in `config.json`. ### Simulate Web Events Run this command to execute the [python](https://www.minusonedb.com/docs/troubleshooting#python-setup) simulator: `python3 simulateClient.py config.json` The script simulates actions users perform on a webpage: arriving at your site, navigating to pages, and leaving the site. Heartbeat events are also simulated; they will be sent to a session store if you have configured one. Otherwise heartbeats will be archived in your environment like all other events. #### Simulate Many Web Events > See heartbeats in the session store: ``` m1 envName-accountName session/query -store stash -q "*" ``` ``` curl https://envName-accountName.minusonedb.com/session/query \ -d "store=stash&q=*" \ -H "m1-auth-token: $myToken" ``` > See the 5 most recent events: ``` m1 envName-accountName query -store index -q '*' -sort "_m1.receivedUTC desc" -rows 5 ``` ``` curl https://envName-accountName.minusonedb.com/query \ -d "store=index&q=*&sort=_m1.receivedUTC desc&rows=5" \ -H "m1-auth-token: $myToken" ``` > See events by state: ``` m1 envName-accountName query -store index -q '*' -json '{ "facet": { "by_state": { "field": "_m1.geo.subdivision_1_iso_code", "type": "terms" } } }' ``` ``` curl https://envName-accountName.minusonedb.com/query -d 'store=index&q=*&json={ "facet": { "by_state": { "field": "_m1.geo.subdivision_1_iso_code", "type": "terms" } } }' -H "m1-auth-token: $myToken" ``` To simulate N concurrent users interacting with a webste, run the following: _Step 1:_`./runSim config.json N` _Step 2:_ Running `tail -f *.log` will give you a view into what is happening with your simulated clients. _Step 3:_ When you are done simulating, you can stop the running simulateClient processes by running `./stopSim` _Step 4:_ (If using a Session Store) Query for heartbeats in the session store. You should see a document representing your active session. _Step 5:_ You can query persisted events using our [reporting](https://reports.minusonedb.com/) application or by using the sample queries below. -------------------------------------------------------------------------------- > Source: https://www.minusonedb.com/docs/samples # General purpose samples Data analytics samples show a number of common data loading and management patterns when using MinusOneDB. Learn how to load large amounts of data as quickly as possible or how to resize stores as your usage grows. ## Load Sample Data Load 50 million Reddit comments in an hour. **_Setup Time:_** 15 minutes **_Data Load Time:_** 40 minutes This python sample shows how to load 50 million Reddit comments into a MinusOneDB environment as quickly as possible. It illustrates common MinusOneDB patterns to maximize loading performance that you can use when you load your own data. Loading larger datasets works in the same way; with a corresponding increase in provisioned hardware, any dataset can be predictably loaded in the same amount of time. Once loaded, explore the sample Reddit dataset using the [m1 client](https://www.minusonedb.com/docs/m1-client) or our [reporting](https://reports.minusonedb.com/) application. ### Prerequisites _Step 1:_ Download and unzip the [reddit loading](https://docs.minusonedb.com/assets/bulkload-reddit.zip) sample. _Step 2:_ Set up the [m1 client](https://www.minusonedb.com/docs/m1-client) if you have not already done so. _Step 3:_ You will need a MinusOneDB environment: - If you do not have a MinusOneDB account, [reach out](mailto:support@minusonedb.com) and we can help you set one up. - If you have a MinusOneDB account but have not set up an environment yet, [create](https://www.minusonedb.com/docs/m1-client#creating-an-environment) one using the `minimal.json` environment template file available in your download. - If you already have your target environment follow the instructions below to run the sample. **Note**: Because this sample uses more than 1 data processing server, it will not work with our [trial](https://www.minusonedb.com/docs/free-trial) environments. Contact [support@minusonedb.com](mailto:support@minusonedb.com) if you want to try this before becoming a MinusOneDB customer. _Step 4:_ Install and configure [python](https://www.minusonedb.com/docs/troubleshooting#python-setup) if you have not already done so. ### Instructions > For reference, here is a copy of ops.json.sample ``` { "server" : "https://ops.minusonedb.com", "username" : "", "password" : "" } ``` > For reference, here is a copy of env.json.sample ``` { "server" : "https://envName-accountName.minusonedb.com", "username" : "", "password" : "" } ``` > Your environment key can be found using the following: > ` > m1 ops env/list -account accountId > ` _Step 1:_ Navigate to the `bulkload-reddit/src` sample directory. _Step 2:_ Copy `ops.json.sample` to `ops.json` and update `ops.json` with your ops.minusonedb.com username and password. _Step 3:_ Copy `env.json.sample` to `env.json` and update your environment name, username, and password. _Step 4:_ Run `python3 bulkload-sample.py ops.json env.json` The script will print out progress as it performs the following actions: - [Rescales](https://www.minusonedb.com/docs/ops-environments#env-rescale) your environment to increase the number of available data processing servers. - Creates a high-performance [data store](https://www.minusonedb.com/docs/ops-environments#env-store-create) to increase data throughput. - [Publishes](https://www.minusonedb.com/docs/env-data-lake#publish) a number of gzip compressed jsonl files that contain the raw reddit data. - Creates a [backup](https://www.minusonedb.com/docs/env-data-stores#store-backup-create) of the store after all files have been published. - Creates a standard [data store](https://www.minusonedb.com/docs/ops-environments#env-store-create). - [Restores](https://www.minusonedb.com/docs/env-data-stores#store-backup-restore) the created backup to the newly created store. - [Destroys](https://www.minusonedb.com/docs/ops-environments#env-store-destroy) the high-performance data store and [drops](https://www.minusonedb.com/docs/env-data-stores#store-drop) the associated configuration. > See the number of Reddit comments: ``` m1 query -store index -q "*" ``` ``` curl https://.minusonedb.com/query \ -d "store=index&q=*" \ -H "m1-auth-token: $myToken" ``` > See the top 10 most popular subreddits: ``` m1 query -store index -q "*" -json '{ "facet": { "by_score": { "field": "subreddit", "type": "terms", "sort" : "count desc" "limit" : 10, } } }' ``` ``` curl https://.minusonedb.com/query -d 'store=index&q=*&json={ "facet": { "by_score": { "field": "subreddit", "type": "terms", "sort" : "count desc", "limit" : 10 } } }' -H "m1-auth-token: $myToken" ``` You can now explore the Reddit data with the sample queries below or with our [reporting](https://reports.minusonedb.com/) application. -------------------------------------------------------------------------------- > Source: https://www.minusonedb.com/docs/best-practices # Data Modeling & Best Practices MinusOneDB stores schema-defined, queryable records that can also carry arbitrary nested JSON. You get the most out of it by modeling *for* its query engine rather than against it. This page collects the patterns that matter most. ## Denormalize at ingest — there is no cross-document join MinusOneDB has no cross-document JOIN. Every "join" becomes a client-side fetch-and-aggregate: many round trips, large payloads, slow first paint, and a cache layer to maintain. The facet engine is the natural shape for analytics — one faceted query per chart. **Rule: every field a query needs lives directly on the record being queried.** Storage is cheap; repeating a small string (vertical, region, owner, client name) on every fact record costs little and turns each chart into a single facet query. When you design a schema, ask: 1. Will any single view query this record type **and** another? → Denormalize the small side onto the large side at ingest. 2. Will users filter or group by this dimension? → Put it on the fact record. 3. Is it slowly-changing reference data? → Denormalize, and re-ingest the affected records when it changes. Genuinely separate record types are still right for independent entities (users, sessions, audit logs) and for arbitrary-shape JSON that varies per record. The rule is about *analytical dimensions*, not every relationship. A faceted analytics query looks like this: ```js { query: '*', facet: { by_vertical: { type: 'terms', field: 'vertical', limit: 20, facet: { revenue: 'sum(revenue)' } } } } ``` ## Faceting: always set `missing: true` A `terms` facet **silently excludes records that have no value for the faceted field**. The bucket sum becomes a strict subset of the dataset, so per-dimension chart totals never reconcile with the headline total — and the gap is invisible. This bites hardest when blank rates are non-trivial and the blank rows skew toward a few high-value records. Set `missing: true` on every `terms` facet that drives a chart, and read the rollup from the sibling `facet.missing` key (it is **not** in the `buckets` array): ```js by_advertiser: { type: 'terms', field: 'advertiser_name', limit: 20, missing: true, facet: { rev: 'sum(revenue)' } } ``` ```js function bucketsWithMissing(facet, label) { if (!facet) return []; const buckets = (facet.buckets || []).slice(); if (facet.missing && (facet.missing.count || 0) > 0) { buckets.push({ val: label || 'Not Reported', count: facet.missing.count, rev: facet.missing.rev || 0 }); } return buckets; } ``` Use a clear sentinel label (`"Not Reported"` for raw dimensions, `"Unmapped"` for taxonomy fields). **Nested-facet caveat:** when a `terms` facet is nested inside another, the inner `missing` count comes back as `0` even when blank rows exist in the parent bucket. Reconstruct it client-side as `parent.count - sum(child counts)`. ## Per-tenant views: scope from the fact store For multi-tenant dashboards (per-client, per-partner, per-seat): - **Build each tenant's view by filtering the fact store on the tenant key** (e.g. `AND cid:N`). Do **not** derive it from a shared side "catalog" record keyed by an id that is common across tenants — a single catalog record per shared id carries one tenant's labels to everyone who shares that id. Consult a catalog only for genuinely id-intrinsic, non-tenant data (e.g. pricing). - **A denormalized field derived from another field must be re-derived whenever its source changes**, or it goes stale and silently mis-attributes. Run a consistency pass on ingest that re-derives the dependent field wherever it disagrees with its source. ## Updating data: use `/modify` - The **data lake is the canonical store** and the target of all CRUD operations. - **`/modify` is the right call for any change** — it does insert, update, and delete in one call. On update, unspecified fields are preserved, so a patch can be minimal. - **`/write` / `/publish` are for archival and bulk load, not for editing live data.** Reaching for them to change existing records is the wrong tool. - **Prefer update over delete.** Delete-and-reload produces phantom/duplicate state; update in place instead. Reserve `/delete` for genuinely removing a record. `POST /modify` with form params: - `e` — JSON array of entities. Each entity needs `_record_type`, plus `_m1key` (integer) **for updates** — without it the entity is treated as an insert. Set only the fields you want to change. - `deleted` — optional JSON array of `_m1key` values to remove. - `publish` — `"true"` (default) for a synchronous index publish. Returns `{firstInserted, updates, deletes}`. ``` # Change one field on one row, leave everything else untouched POST /modify publish=true e=[{"_m1key": 1549520001, "_record_type": "fact_table", "owner": "Acme"}] → {"firstInserted": null, "updates": 1, "deletes": 0} ``` **OR-chunk pattern** for imports keyed by an external id (when you don't have `_m1key` yet): chunk the external ids (~1000 per query), OR-query the fact store (`q=external_id:(id1 OR id2 OR ...)`) to get each row's `_m1key`, build a `{_m1key, _record_type, new_field}` patch per row (skip rows already correct), and `POST /modify` in batches. ## Bulk delete: commit once, at the end When deleting many records, **commit the index a single time after the last batch.** Every committed write is a heavy index-segment flush; committing per batch turns one delete into dozens of flushes. Collect all target `_m1key`s in one query (`fl=_m1key`), then delete carrying every id with one final commit. Browser clients using multipart `FormData` hit a hard cap of 9,999 parts per request (≥10,000 returns HTTP 413), so chunk under that — but still set `commit=false` on every chunk except the last. (Per the doctrine above, still prefer `/modify` for anything that's really an update; this is for genuine bulk removal.) ## Schema field types: choose carefully — they are effectively permanent - **`string`** is a Solr `StrField` with a hard **~32 KB cap** per value; a write over it returns HTTP 400 (`field … which is too large`). Use it for ids, names, enums, and short strings you'll facet on. - **`text`** is a Solr `TextField` and holds large payloads — session recordings, HTML, long JSON, base64. Use it for anything that might exceed a few KB. (The `large: true` flag is not always honoured; `type: text` is the reliable way to get a big-text field.) - Numeric, date, and boolean fields should use the matching typed field — range queries and cost math depend on it. **Type choices are nearly permanent.** Re-adding an existing field name returns HTTP 500 (`duplicate key`), and there is no `/schema/remove` endpoint. If you created a field with the wrong type, don't try to fix it in place: add a *new* field with a new name and the correct type, route writes there, and have readers fall back to whichever field is populated (`fl: new_field,old_field`). The orphaned field is harmless. Choose up front by asking **"could this value ever exceed ~32 KB?"** — yes → `text`; no and you'll facet on it → `string`; numeric/date/boolean → the matching typed field. -------------------------------------------------------------------------------- > Source: https://www.minusonedb.com/docs/loading-data # Loading Data Loading a fresh environment has a few sharp edges that can cause **silent data loss** — publishes that report success while storing nothing useful. This page covers how to load correctly and how to verify you actually did. ## `publish-permissive` controls whether unknown columns are accepted A freshly provisioned environment starts with `publish-permissive: false`. In that state `/publish` rejects any column that doesn't already have a schema field, returning HTTP 400 (`No such property: `). Turning the setting on lets those columns through — but, as the section below explains, that does **not** make them queryable on its own; you still declare the fields you intend to query. ```bash # Check it curl -s "https://.minusonedb.com/system" -H "m1-auth-token: $TOK" | jq '.["publish-permissive"]' # Turn it on curl -s -X POST "https://.minusonedb.com/system" \ -H "m1-auth-token: $TOK" \ --data-urlencode "publish-permissive=true" ``` Cluster propagation can lag by a moment — re-check with the GET if it still reads `false` immediately after. ## `/publish` returns HTTP 200 with the real status in the body `/publish` always returns HTTP 200 at the transport layer. The **actual** status is inside the JSON body: ```json { "path": "/publish", "status": 400, "message": "No such property: parent_name", "error": "Bad Request" } ``` A loader that checks only the HTTP code will classify these as success, fill its state with bogus "ok" records, and leave you with `numFound: 0` and no signal that anything went wrong. **Always check the body status** (and the presence of `error`/`message`), not just the HTTP code. ## Permissive accepts unknown fields — but drops their values This is the most important trap. `publish-permissive: true` **accepts** a publish containing unknown fields, but it does **not** add them to the schema as queryable typed properties. The unknown field *names* are recorded in the multi-string `_m1.unknownProperties` field on each doc; the field **values are silently dropped**. The publish looks successful — HTTP 200, climbing "Saved N total" counts — but a `GET /query?q=*&rows=1` reveals docs containing only `_m1key`, `_m1rand`, `_m1.source`, and `_m1.unknownProperties`: metadata with no data. **When loading data with columns you intend to query, mirror the schema first.** A fresh environment has only a handful of fields; a long-lived one may have a hundred or more, and auto-discovery does not bridge the gap. ```bash # Pull a working env's schema, strip the universal keys, apply to the destination curl -s "https://.minusonedb.com/schema?store=" -H "m1-auth-token: $SRC_TOK" | jq . > schema.json jq 'to_entries | map(select(.key | test("^_m1key$|^_m1rand$") | not) | .value | {name, type, multi: (.multi // false)})' schema.json > properties.json curl -s -X POST "https://.minusonedb.com/schema/add" \ -H "m1-auth-token: $DST_TOK" \ --data-urlencode "store=" \ --data-urlencode "properties@properties.json" ``` ## Declare the `_m1.*` system fields MinusOneDB stamps a cluster of system fields on every published record. They must be present in the schema, or the first chunked publish fails with HTTP 400 (`unknown field '_m1.partial'`). A long-lived environment usually has them; a freshly initialized one does not — add them as a group: | Field | Type | Multi | Purpose | |---|---|---|---| | `_m1.partial` | boolean | — | set on records split across a chunked publish | | `_m1.source` | string | — | archive URI of the publish that produced the record | | `_m1.receivedUTC` | date | — | server-side receive timestamp | | `_m1.ip` | string | — | source IP that submitted the write | | `_m1.errorProperties` | string | yes | property names that failed validation | | `_m1.unknownProperties` | string | yes | property names not in the schema (values dropped under permissive) | Add `_m1.geo.*` fields only when the geo feature is enabled and a geo database is loaded. ## Pre-load checklist Before launching a sustained load into a fresh environment: 1. **Mirror the source schema first** (pull `/schema`, strip `_m1key`/`_m1rand`, apply via `/schema/add`). Non-negotiable if your data has columns you want to query — without it, see the silent-drop trap above. 2. `GET /system` → confirm `publish-permissive: true`. 3. `GET /store/list` → confirm the target store exists. 4. `GET /schema?store=` → confirm the `_m1.*` system fields **and** your user fields exist; diff against the source schema. 5. `GET /query?q=*&rows=0&store=` → confirm the expected baseline (usually `numFound: 0`). 6. Send a **single** small `/publish`, then **fetch one doc back** (`/query?q=*&rows=1&fl=*`) and confirm it has your user-defined fields populated — not just `_m1key` and `_m1.unknownProperties`. This read-back is the only reliable way to catch a silent schema miss; body-status alone shows success even when every value was dropped. Only after that read-back returns real data in real fields should you start the full load. -------------------------------------------------------------------------------- > Source: https://www.minusonedb.com/docs/environment-api # Environment API The services documented below let you interact with one of your MinusOneDB environments. All of the service endpoints described can be accessed via conventional HTTP clients (curl, postman) or programmatically in your desired language or framework. We offer the [m1 client](https://www.minusonedb.com/docs/m1-client) which simplifies credential management and environment interaction. Services use typical HTTP response codes. **2xx:** The request operation worked as as expected. **401:** The token used for the request was invalid. **403:** The user associated with the token used for the request does not have sufficient [rights](https://www.minusonedb.com/docs/ops-rights) to perform the requested operation. **500:** The request could not be completed -------------------------------------------------------------------------------- > Source: https://www.minusonedb.com/docs/env-auth-access # Authentication and Access ### /auth Log in and retrieve token for service access. By default, tokens must be used from the machine they were generated and will expire after a user defined [time](https://www.minusonedb.com/docs/env-system-config#system-token-expire-ms). - **rights:** none, admin if ip or ttl is set - **verbs:** POST | Parameter | Type | Required | |-----------|------|----------| | username | String | Yes | | password | String | Yes | | ip | String | No. Used to log in on behalf of a user at specified ip address. Users with admin rights can use "..." here to generate a token that can be used from any ip address. | | ttl | long | No. Will set a custom expiration time in ms. The default ttl is defined by the [token-expire-ms](https://www.minusonedb.com/docs/env-system-config#system-token-expire-ms) system parameter. | ::: code-group ```bash [curl] curl https://test-m1.minusonedb.com/auth \ -d "username=admin&password=passphrase" ``` ```bash [m1 client] m1 auth test-m1 ``` ::: 💡 Pro Tip: Save your token to a variable and reuse it across API calls — no need to copy and paste it each time: ::: code-group ```bash [curl] myToken=$(curl -X POST https://test-m1.minusonedb.com/auth -d "username=username&password=password&bucket=bucketName") ``` ::: ### /user/password Change user password. - **rights:** none, admin if setting on behalf of another user. - **verbs:** POST | Parameter | Type | Required | |-----------|------|----------| | password | String | Yes. New password. | | username | String | No. Specify user if admin is changing another user's password. | ::: code-group ```bash [curl] curl https://test-m1.minusonedb.com/user/password \ -d "password=newpassphrase" \ -H "m1-auth-token: $myToken" ``` ```bash [m1 client] m1 test-m1 user/password -password "newpassphrase" ``` ::: > No response when successful. -------------------------------------------------------------------------------- > Source: https://www.minusonedb.com/docs/env-users-tokens # Users and Tokens ### /user/list List all users and associated metadata. - **rights:** admin - **verbs:** GET - **parameters:** none ::: code-group ```bash [curl] curl https://test-m1.minusonedb.com/user/list \ -H "m1-auth-token: $myToken" ``` ```bash [m1 client] m1 test-m1 user/list ``` ::: ### /user/get Retrieve metadata for specified user. - **rights:** admin - **verbs:** GET | Parameter | Type | Required | |-----------|------|----------| | username | String | Yes | ::: code-group ```bash [curl] curl https://test-m1.minusonedb.com/user/get?username=user \ -H "m1-auth-token: $myToken" ``` ```bash [m1 client] m1 test-m1 user/get -username user ``` ::: ### /user/add Add new user and specify rights. - **rights:** admin - **verbs:** POST | Parameter | Type | Required | |-----------|------|----------| | username | String | Yes. Must be unique. | | password | String | Yes | | rights | Array | Yes. List of rights to grant to user. | ::: code-group ```bash [curl] curl https://test-m1.minusonedb.com/user/add \ -d 'username=newUser&password=passphrase&rights=schema,get' \ -H "m1-auth-token: $myToken" ``` ```bash [m1 client] m1 test-m1 user/add -username newUser \ -password passphrase -rights '["schema","get"]' ``` ::: > No response when successful. ### /user/update Update user metadata. - **rights:** admin - **verbs:** POST | Parameter | Type | Required | |-----------|------|----------| | username | String | Yes | | rights | Array | Yes. List of rights to grant to user. | ::: code-group ```bash [curl] curl https://test-m1.minusonedb.com/user/update \ -d 'username=user&rights=get,publish' \ -H "m1-auth-token: $myToken" ``` ```bash [m1 client] m1 test-m1 user/update -username user -rights '["get", "publish"]' ``` ::: > No response when successful. ### /user/remove Remove user; the user will no longer be able to access the associated environment. - **rights:** admin - **verbs:** POST | Parameter | Type | Required | |-----------|------|----------| | username | String | Yes | ::: code-group ```bash [curl] curl https://test-m1.minusonedb.com/user/remove \ -d "username=user" \ -H "m1-auth-token: $myToken" ``` ```bash [m1 client] m1 test-m1 user/remove -username user ``` ::: > No response when successful. -------------------------------------------------------------------------------- > Source: https://www.minusonedb.com/docs/env-schema # Schema ### /schema Retrieves current schema. - **rights:** schema, get, publish - **verbs:** GET - **parameters:** none - **returns:** JSON \[{},...\] representing the list of properties that make up the environment schema. ::: code-group ```bash [curl] curl https://test-m1.minusonedb.com/schema \ -H "m1-auth-token: $myToken" ``` ```bash [m1 client] m1 test-m1 schema ``` ::: ### /schema/add Add properties to the environment schema. Can be called as many times as desired to add new property definitions. - **rights:** schema - **verbs:** POST | Parameter | Type | Required | |-----------|------|----------| | properties | property[] | Yes | Each property is a map with the following keys: | Key | Type | Required | |-----|------|----------| | name | String | Yes | | type | String | Yes. One of: string, text, integer, double, date. | | multi | boolean | No. Default false. If true the property can contain multiple values. | | description | String | No. Default null. Comment describing property, purely for informational purposes. | ::: code-group ```bash [curl] curl https://test-m1.minusonedb.com/schema/add \ -d 'properties=[ { "name": "myTags", "type": "string", "multi": true, "description": "Represents important tags." }, { "name": "myBoolean", "type": "boolean" } ]' \ -H "m1-auth-token: $myToken" ``` ```bash [m1 client] m1 test-m1 schema/add -properties '[ { "name": "myTags", "type": "string", "multi": true, "description": "Represents important tags." }, { "name": "myBoolean", "type": "boolean" } ]' # Or load from file: m1 test-m1 schema/add -properties @fileWithSchema.json ``` ::: > No response when successful. ### /schema/wipe Deletes all user defined properties in the environment schema. Note that this does not modify data that has already been written to the data lake. - **rights:** schema - **verbs:** POST - **parameters:** none ::: code-group ```bash [curl] curl -X POST https://test-m1.minusonedb.com/schema/wipe \ -H "m1-auth-token: $myToken" ``` ```bash [m1 client] m1 test-m1 schema/wipe/ ``` ::: > No response when successful. -------------------------------------------------------------------------------- > Source: https://www.minusonedb.com/docs/env-archive # Archive Please note that the only data access to data stored in the Archive layer is via s3 file access, typically by specifying files as the file for a call to [publish](https://www.minusonedb.com/docs/env-data-lake#publish). ### /write Save data to the archive. - **rights:** publish - **verbs:** POST | Parameter | Type | Required | |-----------|------|----------| | items | JSON map[] | Yes. Each item must be a JSON map that can contain arbitrary keys and structure. | | publish | boolean | No. When true, will attempt to publish data to the data lake after archiving it. | | async | boolean | No. Default true. Set to false to receive synchronous error messages in a debugging context. | | rules | TransformRuleSet | No. Used only when publish is true. | ::: code-group ```bash [curl] curl https://test-m1.minusonedb.com/write \ -d 'items=[{"property1" : {"ip" : "127.0.0.1"}, "session" : null, "event": {"id": "key98765", "time": 946702800, "type": "sessionStart"}}]&publish=true' \ -H "m1-auth-token: $myToken" ``` ```bash [m1 client] m1 test-m1 write \ -items '[{"property1":{"ip":"127.0.0.1"},"session":null,"event":{"id":"key98765","time":946702800,"type":"sessionStart"}}]' \ -publish true ``` ::: > No response when successful. In a production setting, if you are publishing data immediately after archiving, we recommend you set [publish-permissive](https://www.minusonedb.com/docs/env-system-config#system-publish-permissive) to true. Each archived item will have the origin ip address and reception time appended to it. See [property-ip](https://www.minusonedb.com/docs/env-system-config#system-property-ip) and [property-received](https://www.minusonedb.com/docs/env-system-config#system-property-received) system parameter documentation for more details. When [geo](https://www.minusonedb.com/docs/env-system-config#system-geo) is enabled, geographic metadata will be appended to each item based on the available [ip address](https://www.minusonedb.com/docs/env-system-config#system-property-ip). -------------------------------------------------------------------------------- > Source: https://www.minusonedb.com/docs/env-data-lake # Data Lake ### /get Retrieve any number of rows from the data lake via `_m1key` property. - **rights:** get - **verbs:** GET, POST | Parameter | Type | Required | |-----------|------|----------| | ids | long[] | Yes. IDs of records to be retrieved. | | properties | Array | No. List of properties from schema to include in records. If null, all columns are returned. | - **returns:** JSON \[{},...\] ::: code-group ```bash [curl] curl https://test-m1.minusonedb.com/get \ -d 'ids=[10000,20000,30000]&properties=["_m1key","session.id"]' \ -H "m1-auth-token: $myToken" ``` ```bash [m1 client] m1 test-m1 get -ids "[10000,20000,30000]" -properties '["_m1key","session.id"]' ``` ::: ### /range Retrieve all rows from datalake with `_m1key` values between `start` (inclusive) and `end` (exclusive). - **rights:** get - **verbs:** GET, POST | Parameter | Type | Required | |-----------|------|----------| | start | long | Yes. Inclusive. | | end | long | Yes. Exclusive. | | properties | Array | No. List of properties from schema to include in records. If null, all columns are returned. | - **returns:** JSON \[{},...\] ::: code-group ```bash [curl] curl https://test-m1.minusonedb.com/range \ -d 'start=10000&end=30000&properties=["_m1key","session.id"]' \ -H "m1-auth-token: $myToken" ``` ```bash [m1 client] m1 test-m1 range -start 10000 -end 30000 ``` ::: ### /publish Load data from files in S3 or GCS into the data lake. - **rights:** publish - **verbs:** POST | Parameter | Type | Required | |-----------|------|----------| | file | String | Yes. Fully qualified file path, e.g. `s3://yourbucket/path/to/file.json` or `gs://yourgcsbucket/path/to/file.json`. | | rules | TransformRuleSet | No | | format | String | No. One of: txt, csv, json, jsonl. If not present, filename will be inspected for format. | | compression | String | No. One of: gz, zip. If not present, filename will be inspected. | | encoding | String | No. Any Java encoding format. Default UTF-8. | | delimiter | char | No. Character for delimited file formats. Tabs and commas assumed for txt/csv unless specified. | | headers | boolean | No. Whether headers are present for delimited formats. Default true. If false, a TransformRuleSet is required. | | rows | Integer | No. Max rows to load. Negative or omitted loads all rows. | ::: code-group ```bash [curl] curl https://test-m1.minusonedb.com/publish \ -d "file=s3://m1-public/reddit/us.jsonl.gz" \ -H "m1-auth-token: $myToken" ``` ```bash [m1 client] m1 test-m1 publish -file "file=s3://m1-public/reddit/us.jsonl.gz" ``` ::: To load data via /publish you must configure permissions so that the EC2 instance profile role associated with your environment can read the file(s) you are attempting to /publish. ```json // Example ReadAccess policy for S3 Configuration { "Version": "2012-10-17", "Statement": [ { "Sid": "ReadAccess", "Effect": "Allow", "Principal": { "AWS": "arn:aws:iam::248899197673:role/$instanceProfileRole" }, "Action": [ "s3:GetObject", "s3:ListBucket" ], "Resource": [ "arn:aws:s3:::$yourBucket", "arn:aws:s3:::$yourBucket/*" ] } ] } ``` #### Configuration Steps When Publishing from S3 _1._ Run [/env/bucket/register](https://www.minusonedb.com/docs/ops-api#env-bucket-register) to register the bucket with your environment _2._ In your AWS account, configure a bucket policy so that the instance profile role of your environment has read access to your data. You can obtain the instanceProfileRole from [/env/get](https://www.minusonedb.com/docs/ops-api#env-get) _3._ Validate your bucket access by calling /publish with one of your files and rows=0. If you get an AccessDeniedException, recheck your IAM permissions above. _4._ You can now call /publish on all the files you wish to load into your environment. ```bash # You can use an existing service account or create one. Skip this step if you already have a service account. # You can replace the account name and display name with anything you like gcloud iam service-accounts create m1db-archive-bucket-reader --display-name="m1db archive bucket reader" # You will need the email address of the account you just created. You can find it in the console (under IAM > Service Accounts) # Or you can run the below replacing the email filter with the name you gave your service account export SA_EMAIL=$(gcloud iam service-accounts list \ --filter="email:m1db-archive-bucket-reader" \ --format="value(email)") # Replace with your bucket name gcloud storage buckets add-iam-policy-binding gs:// \ --member="serviceAccount:$SA_EMAIL" \ --role="roles/storage.objectViewer" ``` #### Configuration Steps When Publishing from GCS _1._ Create a service account in your google cloud account and allow it to read/download files in your bucket(s) (You can skip the step if you already have a service account that has sufficient access to the data you wish to load in your m1db environment) _2._ Create a trust relationship between the EC2 instance profile role associated with your environment and your service account Select a configuration that refers to the project containing your GCS bucket. Alternatively you can add --configuration <config> to each of the commands below The account in the <config> must have at least these permissions on the relevant project: - iam.serviceAccounts.create - iam.serviceAccounts.setIamPolicy - storage.buckets.setIamPolicy - gcloud config configurations activate <config> ```bash # Replace with your environment name export M1_INSTANCE_PROFILE=$(m1 ops env/get -env |grep instanceProfileRole|sed -e 's/\(.*\): "\(.*\)",/\2/') # Create a trust relationship between your service account and your environment instance role # Note that this role is mediated by an m1db GCP account. gcloud iam service-accounts add-iam-policy-binding $SA_EMAIL \ --role=roles/iam.workloadIdentityUser \ --member="principalSet://iam.googleapis.com/projects/980494489932/locations/global/workloadIdentityPools/aws-pool/attribute.aws_role/arn:aws:sts::248899197673:assumed-role/$M1_INSTANCE_PROFILE" ``` _3._ Set the [gcs-service-account](https://www.minusonedb.com/docs/env-system-config#system-gcs-service-account) system property to the email address of your service account. ```bash # Set the gcs-service-account system property to point to your service account m1 system -gcs-service-account $SA_EMAIL ``` _4._ [Enable](https://www.minusonedb.com/docs/ops-api#env-outbound) outbound connectivity for your environment ```bash # Enable outbound connectivity for your environment m1 ops env/outbound -env -enable true ``` _5._ Validate your GCS bucket access by calling /publish with one of your files and rows=0. If you get an access error, recheck your configuration steps. Note that it may take a few minutes for your configuration to take effect. _6._ You can now call /publish on all the files you wish to load into your environment. ### /insert Insert raw entities into the lake. - **rights:** publish - **verbs:** POST | Parameter | Type | Required | |-----------|------|----------| | e | String[] | No. Raw parameter entities | - **returns:** JSON \[{},...\] ::: code-group ```bash [curl] curl https://test-m1.minusonedb.com/insert \ -d 'e=[{"score" : "199", "downs" : "10", "author" : "Alice"}]' -H "m1-auth-token: $myToken" ``` ```bash [m1 client] m1 test-m1 insert -e "[{"score" : "199", "downs" : "10", "author" : "Alice"}]" ``` ::: ### /update Update documents in lake by passing in entities associated with their _m1key. Beware of bulk updates across many different files, updating many documents at once will take much longer. - **rights:** delete - **verbs:** POST | Parameter | Type | Required | |-----------|------|----------| | e | String[] | No. Raw parameter entities | - **returns:** JSON \[{},...\] ::: code-group ```bash [curl] curl https://test-m1.minusonedb.com/update \ -d 'e=[{"_m1key" : "82270000", "score" : "199", "downs" : "10", "author" : "Alice"}]' -H "m1-auth-token: $myToken" ``` ```bash [m1 client] m1 test-m1 update -e "[{"_m1key" : "82270000", "score" : "199", "downs" : "10", "author" : "Alice"}]" ``` ::: ### /modify Modify datalake with inserts, updates or deletions. This method is a generalization of /insert, /update, /delete. Beware of bulk updates across many different files, updating many documents at once will take much longer. - **rights:** publish - **verbs:** POST | Parameter | Type | Required | |-----------|------|----------| | e | String[] | No. Raw parameter entities | | delete | String[] | No. Raw ids to delete | - **returns:** JSON \[{},...\] ::: code-group ```bash [curl] curl https://test-m1.minusonedb.com/modify \ -d 'e=[{"score":"199","downs":"10","author":"Alice"}]&delete=["ids=82270000"]' \ -H "m1-auth-token: $myToken" ``` ```bash [m1 client] m1 test-m1 modify -e '[{"score" : "199", "downs" : "10", "author" : "Alice"}]' -delete "["ids=82270000"]" ``` ::: ### /delete Delete _m1key records from the datalake for the specified list of m1keys. - **rights:** delete - **verbs:** POST | Parameter | Type | Required | |-----------|------|----------| | ids | String[] | No. List of m1key ids to be deleted | - **returns:** JSON \[{},...\] ::: code-group ```bash [curl] curl https://test-m1.minusonedb.com/delete \ -d "ids=["82270000"]" \ -H "m1-auth-token: $myToken" ``` ```bash [m1 client] m1 test-m1 delete -ids '["82270000"]' ``` ::: ### /next Retrieve the next `_m1key` that will be assigned to a document added to the lake (via [/publish](#publish), for example). - **rights:** admin, publish - **verbs:** GET - **parameters:** none ::: code-group ```bash [curl] curl https://test-m1.minusonedb.com/next \ -H "m1-auth-token: $myToken" ``` ```bash [m1 client] m1 test-m1 next ``` ::: -------------------------------------------------------------------------------- > Source: https://www.minusonedb.com/docs/env-data-stores # Data Stores ### /query Queries the specified data store. - **rights:** get - **verbs:** GET, POST | Parameter | Type | Required | Description | |-----------|------|----------|-------------| | store | String | Yes | Names of the data stores to query | | q | String | Yes | Query string as described below | | fl | String | No | Comma separated list of [property names](https://www.minusonedb.com/docs/environment-api#schema-1) to return for each matching record. Default returns all properties | | start | long | No | The starting record to return. Default is 0 | | rows | long | No | Number of records to return. Default will return 0 records | | sort | String | No | Order in which to return records. A sort string is made of comma separated parts where each part is a [property name](https://www.minusonedb.com/docs/environment-api#schema-1) followed by one of `asc` or `desc` | | format | String | No | One of `csv` or `json`. Default is `json`. For csv, both documents and facets cannot be returned in 1 request | The easiest way to learn the query is by studying the following simple examples. > Return all documents: ::: code-group ```bash [curl] curl https://test-m1.minusonedb.com/query \ -d "store=index&q=*&rows=10" \ -H "m1-auth-token: $myToken" ``` ```bash [m1 client] m1 test-m1 query -store "index" -q '*' -rows 10 ``` ::: The simplest query is `*` which will match all documents. Add a rows parameter to get data other than a count. > All events from a particular IP address: ```bash m1 test-m1 query -store "index" -q "_m1.ip:128.218.229.26" ``` ```bash curl https://test-m1.minusonedb.com/query \ -d "store=index&q=_m1.ip:128.218.229.26" \ -H "m1-auth-token: $myToken" ``` To find all events from a particular ip address use a query like `_m1.ip:128.218.229.26`. In general, the form is `property:value`. > All events from 2 IP addresses: ```bash m1 test-m1 query -store "index" -q '_m1.ip:("128.218.229.26" OR "72.200.1.74")' ``` ```bash curl https://test-m1.minusonedb.com/query \ -d "store=index&q=_m1.ip:("128.218.229.26" OR "72.200.1.74")" \ -H "m1-auth-token: $myToken" ``` To find all events from more than ip address, use `OR` to separate multiple values: `_m1.ip:("128.218.229.26" OR "72.200.1.74")`. > More than 1 constraint: ```bash m1 test-m1 query -store "index" -q "_m1.ip:("128.218.229.26" OR "72.200.1.74") AND event.type:sessionStart" ``` ```bash curl https://test-m1.minusonedb.com/query \ -d "store=index&q=_m1.ip:("128.218.229.26" OR "72.200.1.74") AND event.type:sessionStart" \ -H "m1-auth-token: $myToken" ``` To use multiple constraints, use an `AND` between individual property terms. The query `_m1.ip:(128.218.229.26 OR 72.200.1.74) AND event.type:sessionStart` will return events of type `sessionStart` from 2 the specified ip addresses. > Match against a range of values for a given property: ```bash m1 test-m1 query -store "index" -q "_m1.receivedUTC:[2021-12-25T00:00:00Z TO 2021-12-26T00:00:00}" -sort '_m1.receivedUTC desc' ``` ```bash curl https://test-m1.minusonedb.com/query \ -d "store=index&q=_m1.receivedUTC:[2021-12-25T00:00:00Z TO 2021-12-26T00:00:00}&sort=_m1.receivedUTC desc" \ -H "m1-auth-token: $myToken" ``` Ranges let you search against a numeric or date interval: `q=_m1.receivedUTC:[2021-12-25T00:00:00Z TO 2021-12-26T00:00:00Z}` will return all events received on 12/25/2021. Use `[]` for inclusive ranges and `{}` for exclusive ranges. > A more complicated query: ```bash m1 test-m1 query -store "index" -q '_m1.receivedUTC:[2021-12-25T00:00:00Z TO 2021-12-26T00:00:00Z} AND (_m1.ip:"128.218.229.26") AND NOT (event.type:sessionStart)' ``` ```bash curl https://test-m1.minusonedb.com/query \ -d 'store=index&q=_m1.receivedUTC:[2021-12-25T00:00:00Z TO 2021-12-26T00:00:00Z} AND (_m1.ip:"128.218.229.26") AND NOT (event.type:sessionStart)' \ -H "m1-auth-token: $myToken" ``` These simple concepts can be combined and composed to make more complicated queries. Our [reports interface](https://reports.minusonedb.com/) can help you build queries and display results. ::: info Streaming This is a streaming endpoint. ::: ### /index Writes all rows in the datalake with `_m1key` values between `start` (inclusive) and `end` (exclusive) to the associated store. Often used when resizing a datastore. - **rights:** publish - **verbs:** POST | Parameter | Type | Required | Description | |-----------|------|----------|-------------| | store | String | Yes | Store name | | start | long | Yes | Inclusive start key | | end | long | Yes | Exclusive end key | ::: code-group ```bash [curl] curl https://test-m1.minusonedb.com/index \ -d "store=index&start=10000&end=30000" \ -H "m1-auth-token: $myToken" ``` ```bash [m1 client] m1 test-m1 index -store index -start 10000 -end 30000 ``` ::: ### /store/list Return a list of all configured data stores. - **rights:** get, admin - **verbs:** GET | Parameter | Type | Required | Description | |-----------|------|----------|-------------| | *(none)* | — | — | — | ::: code-group ```bash [curl] curl https://test-m1.minusonedb.com/store/list \ -H "m1-auth-token: $myToken" ``` ```bash [m1 client] m1 test-m1 store/list ``` ::: ### /store/autocommit Set autocommit frequency of a given store. The autocommit frequency governs how often a store makes recently written data available for searching. For most near real time use cases we recommend a value of 5; a 0 or negative value will turn off autocommit which is recommended for bulk loading use cases. - **rights:** admin - **verbs:** POST | Parameter | Type | Required | Description | |-----------|------|----------|-------------| | store | String | Yes | Store name | | seconds | integer | Yes | Autocommit frequency in seconds | ::: code-group ```bash [curl] curl https://test-m1.minusonedb.com/store/autocommit \ -d "store=index&seconds=2" \ -H "m1-auth-token: $myToken" ``` ```bash [m1 client] m1 test-m1 store/autocommit -store index -seconds 2 ``` ::: ### /store/add Add specification for a data store. You will need a corresponding [/env/store/create](https://www.minusonedb.com/docs/ops-api#env-store-create) ops service call to provision your data store servers. - **rights:** admin - **verbs:** POST | Parameter | Type | Required | Description | |-----------|------|----------|-------------| | store | String | Yes | Store name | | type | String | Yes | One of `standard` or `boost` | | shards | long | Yes | Number of shards in deployed store | | replicas | long | Yes | Number of replicas in deployed store | | percent | double | No | Percentage (0 to 1); store will be sampled to this subset of all published data based on value of `_m1rand` | ::: code-group ```bash [curl] curl https://test-m1.minusonedb.com/store/add \ -d "store=index&shards=1&replicas=2" \ -H "m1-auth-token: $myToken" ``` ```bash [m1 client] m1 test-m1 store/add -store index -shards 1 -replicas 2 ``` ::: The [m1 client](https://www.minusonedb.com/docs/m1-client#adding-stores) can help streamline operations that require coordination between `https://ops.minusonedb.com` and an environment. ### /store/drop Removes the specification for a data store. You will need to make a corresponding [/env/store/destroy](https://www.minusonedb.com/docs/ops-api#env-store-destroy) ops service call so as not to incur charges for provisioned servers. - **rights:** admin - **verbs:** POST | Parameter | Type | Required | Description | |-----------|------|----------|-------------| | store | String | Yes | Store name | ::: code-group ```bash [curl] curl https://test-m1.minusonedb.com/store/drop \ -d "store=index" \ -H "m1-auth-token: $myToken" ``` ```bash [m1 client] m1 test-m1 store/drop -store index ``` ::: The [m1 client](https://www.minusonedb.com/docs/m1-client#dropping-a-store) can help streamline operations that require coordination between `https://ops.minusonedb.com` and an environment. ### /store/rename Change the name of a data store. - **rights:** admin - **verbs:** POST | Parameter | Type | Required | Description | |-----------|------|----------|-------------| | store | String | Yes | Store name | | name | String | Yes | New store name | ::: code-group ```bash [curl] curl https://test-m1.minusonedb.com/store/rename \ -d "store=index&name=newStoreName" \ -H "m1-auth-token: $myToken" ``` ```bash [m1 client] m1 test-m1 store/rename -store index -name newStoreName ``` ::: The [m1 client](https://www.minusonedb.com/docs/m1-client#renaming-a-store) can help streamline operations that require coordination between `https://ops.minusonedb.com` and an environment. ### /store/enable Set the data store active status to true/false. A store set to active will be automatically updated when the data lake is updated; an inactive store will not. - **rights:** admin - **verbs:** POST | Parameter | Type | Required | Description | |-----------|------|----------|-------------| | store | String | Yes | Store name | | active | Boolean | Yes | Active status | ::: code-group ```bash [curl] curl https://test-m1.minusonedb.com/store/enable \ -d "store=index&active=true" \ -H "m1-auth-token: $myToken" ``` ```bash [m1 client] m1 test-m1 store/enable -store index -active true ``` ::: ### /store/backup/create Starts a backup of the store; use [/store/backup/list](#store-backup-list) to monitor status of the backup. - **rights:** admin - **verbs:** POST | Parameter | Type | Required | Description | |-----------|------|----------|-------------| | store | String | Yes | Store name | | name | String | Yes | Backup name | ::: code-group ```bash [curl] curl https://test-m1.minusonedb.com/store/backup/create \ -d "store=index&name=myBackup" \ -H "m1-auth-token: $myToken" ``` ```bash [m1 client] m1 test-m1 store/backup/create -store index -name myBackup ``` ::: ### /store/backup/restore Begins the restore process of a store from a saved backup. Any data that was previously in the store will be lost once the restore process begins. Use [/health](https://www.minusonedb.com/docs/env-operations#health) to monitor status of the restore; the store server group will require a [/reboot](https://www.minusonedb.com/docs/ops-api#env-reboot) to complete the restore process. - **rights:** admin - **verbs:** POST | Parameter | Type | Required | Description | |-----------|------|----------|-------------| | store | String | Yes | Store name | | backup | String | Yes | ID of backup | ::: code-group ```bash [curl] curl https://test-m1.minusonedb.com/store/backup/restore \ -d "store=index&backup=5ed16f63-1282-41b6-a880-67635ce5036a" \ -H "m1-auth-token: $myToken" ``` ```bash [m1 client] m1 test-m1 store/backup/restore -store index -backup 5ed16f63-1282-41b6-a880-67635ce5036a ``` ::: ### /store/backup/list Return a list of all backups associated with this environment. - **rights:** admin - **verbs:** GET | Parameter | Type | Required | Description | |-----------|------|----------|-------------| | *(none)* | — | — | — | ::: code-group ```bash [curl] curl https://test-m1.minusonedb.com/store/backup/list \ -H "m1-auth-token: $myToken" ``` ```bash [m1 client] m1 test-m1 store/backup/list ``` ::: ### /store/backup/drop Remove the specification and all underlying backup data for a data store backup. ::: warning This operation cannot be undone. ::: - **rights:** admin - **verbs:** POST | Parameter | Type | Required | Description | |-----------|------|----------|-------------| | backup | String | Yes | ID of backup | ::: code-group ```bash [curl] curl https://test-m1.minusonedb.com/store/backup/drop \ -d "backup=5ed16f63-1282-41b6-a880-67635ce5036a" \ -H "m1-auth-token: $myToken" ``` ```bash [m1 client] m1 test-m1 store/backup/drop -backup 5ed16f63-1282-41b6-a880-67635ce5036a ``` ::: ### /store/backup/rename Change the name of a store backup. - **rights:** admin - **verbs:** POST | Parameter | Type | Required | Description | |-----------|------|----------|-------------| | backup | String | Yes | ID of backup | | name | String | Yes | New backup name | ::: code-group ```bash [curl] curl https://test-m1.minusonedb.com/store/backup/rename \ -d "backup=5ed16f63-1282-41b6-a880-67635ce5036a&name=newBackupName" \ -H "m1-auth-token: $myToken" ``` ```bash [m1 client] m1 test-m1 store/backup/rename -backup 5ed16f63-1282-41b6-a880-67635ce5036a -name newBackupName ``` ::: -------------------------------------------------------------------------------- > Source: https://www.minusonedb.com/docs/env-session-stores # Session Stores ### /session/update Upsert and/or delete data stored in a specified session store. ::: code-group ```bash [curl] curl https://test-m1.minusonedb.com/session/update -d 'store=stash&ops=[{"upsert":[ { "ip": "65.29.29.190", "session": { "id": "0295aed9-5872-4586-8086-57d3d8c40fb8", "start": 1612497996184 }, "user": { "id": "f7661f25-7c49-4f3b-9185-ec6566892878" }, "event": { "id": "6dd089ce-5a96-4fac-a405-43c1ab3eb6b3", "time": 1612497996000, "type": "heartbeat", "data": { "a": "timers", "b": "string", "foo": "epe", "bar": "string" } } }]}, {"delete" : ["78ba3dac-3e45-4e17-940e-e66584882ce4"]}]' \ -H "m1-auth-token: $myToken" ``` ```bash [m1 client] m1 test-m1 session/update -store stash -ops '[{"upsert": [{"ip": "65.29.29.190","session": {"id": "0295aed9-5872-4586-8086-57d3d8c40fb8","start": 1612497996184}}]},{"delete": ["78ba3dac-3e45-4e17-940e-e66584882ce4"]}]' ``` ::: - **rights:** sessionUpdate - **verbs:** POST | Parameter | Type | Required | |-----------|------|----------| | store | String | Yes | | ops | JSON (op[]) | Yes | Each op may have two properties: `upsert` (map[], must have session id parameter set) and `delete` (Array, session ids to be deleted). > No response when successful. ### /session/query Retrieve matching records from a session store. The syntax and semantics of /session/query are the same as [/query](https://www.minusonedb.com/docs/env-data-stores#query) where possible. Because session stores accept items that have a deeply nested structure, dotted notation is used to signify nesting. In the example below, the archived item contained an `m1` key that was a map containing an `ip` key. To query against that property in /session/query, use `m1.ip` as the property name. ::: code-group ```bash [curl] curl https://test-m1.minusonedb.com/session/query \ -d 'store=stash&q=_m1.ip:64.150.5.135' \ -H "m1-auth-token: $myToken" ``` ```bash [m1 client] m1 test-m1 session/query -store stash -q "_m1.ip:64.150.5.135" ``` ::: - **rights:** sessionQuery - **verbs:** POST | Parameter | Type | Required | |-----------|------|----------| | store | String | Yes | | q | String | Yes | | sort | String | No | | start | long | No | | rows | long | No | - **returns:** JSON map containing a count of items matching the query and the requested number of matching items **Note**: This is a streaming endpoint. ### /session/list Return a list of all configured session stores with metadata. ::: code-group ```bash [curl] curl https://test-m1.minusonedb.com/session/list \ -H "m1-auth-token: $myToken" ``` ```bash [m1 client] m1 test-m1 session/list ``` ::: - **rights:** get, admin - **verbs:** GET - **parameters:** none ### /session/key Set the key to be used for identity tests. ::: code-group ```bash [curl] curl https://test-m1.minusonedb.com/session/key \ -d "store=stash&key=session.id" \ -H "m1-auth-token: $myToken" ``` ```bash [m1 client] m1 test-m1 session/key -store stash -key "session.id" ``` ::: - **rights:** admin - **verbs:** POST | Parameter | Type | Required | |-----------|------|----------| | store | String | Yes | | key | String | Yes | > No response when successful. ### /session/add Add configuration for a newly deployed session store. Called after a new store is [deployed](https://www.minusonedb.com/docs/ops-api#env-session-create) via MinusOneDB ops services. Use the [m1 client](https://www.minusonedb.com/docs/m1-client#adding-stores) to provision hardware and update your environment with one command. ::: code-group ```bash [curl] curl https://test-m1.minusonedb.com/session/add \ -d "store=stash&key=session.id&shards=1&replicas=1" \ -H "m1-auth-token: $myToken" ``` ```bash [m1 client] m1 test-m1 session/add -store stash -key "session.id" -shards 1 -replicas 1 ``` ::: - **rights:** admin - **verbs:** POST | Parameter | Type | Required | |-----------|------|----------| | store | String | Yes | | key | String | No | | shards | long | Yes | | replicas | long | Yes | The `key` defaults to `session.id`. > No response when successful. ### /session/rename Change the name of a session store. ::: code-group ```bash [curl] curl https://test-m1.minusonedb.com/session/rename \ -d "store=stash&name=newSessionStoreName" \ -H "m1-auth-token: $myToken" ``` ```bash [m1 client] m1 test-m1 session/rename -store stash -name newSessionStoreName ``` ::: - **rights:** admin - **verbs:** POST | Parameter | Type | Required | |-----------|------|----------| | store | String | Yes | | name | String | Yes | > No response when successful. ### /session/drop Removes configuration for a session store. Typically called after the store is [destroyed](https://www.minusonedb.com/docs/ops-api#env-session-destroy) via MinusOneDB ops services. Use the [m1 client](https://www.minusonedb.com/docs/m1-client#dropping-a-store) to decommission server resources and update your environment with one command. ::: code-group ```bash [curl] curl https://test-m1.minusonedb.com/session/drop \ -d "store=stash" \ -H "m1-auth-token: $myToken" ``` ```bash [m1 client] m1 test-m1 session/drop -store stash ``` ::: - **rights:** admin - **verbs:** POST | Parameter | Type | Required | |-----------|------|----------| | store | String | Yes | > No response when successful. -------------------------------------------------------------------------------- > Source: https://www.minusonedb.com/docs/env-system-config # System Configuration Each system parameter is described in detail below; a GET of `/system/parameter` will let you see the current value of `parameter` while a POST to the same path will let you change `parameter`'s value. System parameters are extensible for application-specific usecases. ### /system Return all environment system parameters or set multiple system parameters at once. ::: code-group ```bash [curl] curl https://test-m1.minusonedb.com/system/ \ -H "m1-auth-token: $myToken" curl https://test-m1.minusonedb.com/system/ \ -d 'publish-permissive=true&geo=true&cors=["minusonedb.com"]&token-expire-ms=86400000' \ -H "m1-auth-token: $myToken" ``` ```bash [m1 client] m1 test-m1 system m1 test-m1 system -publish-permissive true -geo true -cors '["minusonedb.com"] -token-expire-ms 86400000 ``` ::: - **rights:** admin - **verbs:** GET or POST - **parameters:** none - **returns:** JSON [{},...] ### /system/init Return true if the environment is [initialized](https://www.minusonedb.com/docs/env-operations#init) and false otherwise. Setting this parameter outside of [/init](https://www.minusonedb.com/docs/env-operations#init) is not supported at this time. ::: code-group ```bash [curl] curl https://test-m1.minusonedb.com/system/init \ -H "m1-auth-token: $myToken" ``` ```bash [m1 client] m1 test-m1 system/init ``` ::: - **rights:** admin - **verbs:** GET - **parameters:** none - **returns:** String ### /system/bucket Return the S3 bucket used to store environment data. This is set at [/init](https://www.minusonedb.com/docs/env-operations#init) time and should be the the bucket associated with the environment at [creation](https://www.minusonedb.com/docs/ops-api#env-create) time. Setting this parameter outside of [/init](https://www.minusonedb.com/docs/env-operations#init) is not supported at this time. ::: code-group ```bash [curl] curl https://test-m1.minusonedb.com/system/bucket \ -H "m1-auth-token: $myToken" ``` ```bash [m1 client] m1 test-m1 system/bucket ``` ::: - **rights:** admin - **verbs:** GET - **parameters:** none - **returns:** String ### /system/region Return the AWS region used for this environment's infrastructure. No value other than us-east-1 is currently supported. Setting this parameter is not supported at this time. ::: code-group ```bash [curl] curl https://test-m1.minusonedb.com/system/region \ -H "m1-auth-token: $myToken" ``` ```bash [m1 client] m1 test-m1 system/region ``` ::: - **rights:** admin - **verbs:** GET - **parameters:** none - **returns:** String ### /system/password-min-length Return or set the minimum password length for users of the environment. The default is 8. ::: code-group ```bash [curl] curl https://test-m1.minusonedb.com/system/password-min-length \ -H "m1-auth-token: $myToken" curl https://test-m1.minusonedb.com/system \ -d "password-min-length=8" -H "m1-auth-token: $myToken" ``` ```bash [m1 client] m1 test-m1 system/password-min-length m1 test-m1 system -password-min-length 8 ``` ::: - **rights:** admin - **verbs:** GET or POST - **returns:** String | Parameter | Type | Required | |-----------|------|----------| | value | long | Yes | ### /system/token-expire-ms Return or set the default token expiration time (ttl). The default is 720000000 (200 hours). ::: code-group ```bash [curl] curl https://test-m1.minusonedb.com/system/token-expire-ms \ -H "m1-auth-token: $myToken" curl https://test-m1.minusonedb.com/system \ -d "token-expire-ms=86400000" -H "m1-auth-token: $myToken" ``` ```bash [m1 client] m1 test-m1 system/token-expire-ms m1 test-m1 system -token-expire-ms 86400000 ``` ::: - **rights:** admin - **verbs:** GET or POST - **returns:** String | Parameter | Type | Required | |-----------|------|----------| | value | long | Yes | ### /system/publish-permissive Return or set whether rows that fail schema validation will cause entire blocks of published data to be rejected atomically. If true, rows that fail schema validation will be persisted to the extent possible; all properties that pass validation will be persisted. Rows that are partially persisted will have have their [property-partial](#system-property-partial) property set to true. If false, attempts to publish rows that fail schema validation will fail with an error. The default value is false. ::: code-group ```bash [curl] curl https://test-m1.minusonedb.com/system/publish-permissive \ -H "m1-auth-token: $myToken" curl https://test-m1.minusonedb.com/system \ -d "publish-permissive=true" -H "m1-auth-token: $myToken" ``` ```bash [m1 client] m1 test-m1 system/publish-permissive m1 test-m1 system -publish-permissive true ``` ::: - **rights:** admin - **verbs:** GET or POST - **parameters:** none - **returns:** String ### /system/property-partial Return or set the property used to signal when rows are only partially persisted. This is only effectively used when [publish-permissive](#system-publish-permissive) is set to true. The specified property must be available in the environment [schema](https://www.minusonedb.com/docs/environment-api#schema). The default value is `_m1.partial`. ::: code-group ```bash [curl] curl https://test-m1.minusonedb.com/system/property-partial \ -H "m1-auth-token: $myToken" curl https://test-m1.minusonedb.com/system \ -d "property-partial=_m1.partial" -H "m1-auth-token: $myToken" ``` ```bash [m1 client] m1 test-m1 system/property-partial m1 test-m1 system -property-partial "_m1.partial" ``` ::: - **rights:** admin - **verbs:** GET or POST - **returns:** String | Parameter | Type | Required | |-----------|------|----------| | value | String | Yes | ### /system/property-ip Return or set the property name where the ip address of the request origin machine will be added to items sent to the [archive layer](https://www.minusonedb.com/docs/environment-api#write) or [session store](https://www.minusonedb.com/docs/env-session-stores#session-update). Items that already contain a non-null value in this property will be left unchanged. Additionally, data in this property will be used for ip address based geocoding if [geo](#system-geo) is suitably configured. The specified property must be available in the environment [schema](https://www.minusonedb.com/docs/environment-api#schema). The default value of this property is `_m1.ip`. ::: code-group ```bash [curl] curl https://test-m1.minusonedb.com/system/property-ip \ -H "m1-auth-token: $myToken" curl https://test-m1.minusonedb.com/system \ -d "property-ip=_m1.ip" -H "m1-auth-token: $myToken" ``` ```bash [m1 client] m1 test-m1 system/property-ip m1 test-m1 system -property-ip "_m1.ip" ``` ::: - **rights:** admin - **verbs:** GET or POST - **returns:** String | Parameter | Type | Required | |-----------|------|----------| | value | String | Yes | ### /system/property-received Return or set the property name where server receipt time will be added to items sent to the [archive layer](https://www.minusonedb.com/docs/environment-api#write) or [session store](https://www.minusonedb.com/docs/env-session-stores#session-update). The specified property must be available in the environment schema. The default value of this property is `_m1.receivedUTC`. ::: code-group ```bash [curl] curl https://test-m1.minusonedb.com/system/property-received \ -H "m1-auth-token: $myToken" curl https://test-m1.minusonedb.com/system \ -d "property-received=_m1.receivedUTC" -H "m1-auth-token: $myToken" ``` ```bash [m1 client] m1 test-m1 system/property-received m1 test-m1 system -property-received "_m1.receivedUTC" ``` ::: - **rights:** admin - **verbs:** GET or POST - **returns:** String | Parameter | Type | Required | |-----------|------|----------| | value | String | Yes | ### /system/geo Return or specify whether geo data will be appended to items sent to the [archive layer](https://www.minusonedb.com/docs/environment-api#write) or [session store](https://www.minusonedb.com/docs/env-session-stores#session-update). Geographic data is based on ip address. The [property-ip](#system-property-ip) system parameter specifies where to look for the ip address in a to be archived document. The schema of an environment with geo turned on must include the set of geo properties in the environment [schema](https://www.minusonedb.com/docs/environment-api#schema) (which can be mapped to alternate property names as desired). ::: code-group ```bash [curl] curl https://test-m1.minusonedb.com/system/geo \ -H "m1-auth-token: $myToken" curl https://test-m1.minusonedb.com/system \ -d "geo=true" -H "m1-auth-token: $myToken" ``` ```bash [m1 client] m1 test-m1 system/geo m1 test-m1 system -geo true ``` ::: - **rights:** admin - **verbs:** GET or POST - **returns:** String | Parameter | Type | Required | |-----------|------|----------| | value | Boolean | Yes | ### /system/gcs-service-account The google service account you have set up that has access to files you wish to load via [/publish](https://www.minusonedb.com/docs/environment-api#publish) and that you have configured a trust relationship with the instance role of your environment. This is only necessary if you are using [/publish](https://www.minusonedb.com/docs/environment-api#publish) to load data from a gcs bucket. ::: code-group ```bash [curl] curl https://test-m1.minusonedb.com/system/gcs-service-account \ -H "m1-auth-token: $myToken" curl https://test-m1.minusonedb.com/system/gcs-service-account \ -d "value=$yourServiceAccount" -H "m1-auth-token: $myToken" ``` ```bash [m1 client] m1 test-m1 system/gcs-service-account m1 test-m1 system -gcs-service-account $yourServiceAccount ``` ::: - **rights:** admin - **verbs:** GET or POST - **returns:** String | Parameter | Type | Required | |-----------|------|----------| | value | String | Yes | The service account must have the `roles/storage.objectViewer` role for files you wish to load via [/publish](https://www.minusonedb.com/docs/environment-api#publish) into your environment. ### /system/cors Return or set the list of domains that will be accepted for CORS requests; MinusOneDB environment will accept requests from all specifies domains and their subdomains. Use [/system/cors/add](#system-cors-add) and [/system/cors/remove](#system-cors-remove) to add or remove individual domains. ::: code-group ```bash [curl] curl https://test-m1.minusonedb.com/system/cors \ -H "m1-auth-token: $myToken" curl https://test-m1.minusonedb.com/system \ -d 'cors=["minusonedb.com","yourdomain.com"]' -H "m1-auth-token: $myToken" ``` ```bash [m1 client] m1 test-m1 system/cors m1 test-m1 system -cors '["minusonedb.com","yourdomain.com"]' ``` ::: - **rights:** admin - **verbs:** GET or POST - **returns:** String | Parameter | Type | Required | |-----------|------|----------| | value | Array | Yes | ### /system/cors/add Add CORS [domains](#system-cors). ::: code-group ```bash [curl] curl https://test-m1.minusonedb.com/system/cors/add \ -d 'domains=["domain1.com","domain2.com"]' -H "m1-auth-token: $myToken" ``` ```bash [m1 client] m1 test-m1 system/cors/add -domains '["domain1.com","domain2.com"]' ``` ::: - **rights:** admin - **verbs:** POST | Parameter | Type | Required | |-----------|------|----------| | domains | Array | Yes | ### /system/cors/remove Remove CORS [domains](#system-cors). ::: code-group ```bash [curl] curl https://test-m1.minusonedb.com/system/cors/remove \ -d 'domains=["domain1.com","domain2.com"]' -H "m1-auth-token: $myToken" ``` ```bash [m1 client] m1 test-m1 system/cors/remove -domains '["domain1.com","domain2.com"]' ``` ::: - **rights:** admin - **verbs:** POST | Parameter | Type | Required | |-----------|------|----------| | domains | Array | Yes | -------------------------------------------------------------------------------- > Source: https://www.minusonedb.com/docs/env-operations # Operations ### /ping Simple boolean, always returns true in non-error states. Indicates the minimal functioning of the system as a whole. Use [/health](#health) for a deeper diagnostic. ::: code-group ```bash [curl] curl https://test-m1.minusonedb.com/ping ``` ```bash [m1 client] m1 test-m1 ping ``` ::: - **rights:** none - **verbs:** GET - **parameters:** none ### /init A POST will initialize a new environment; this operation can only be performed once. GET calls to /init will return a boolean indicating whether the system has been initialized. ::: code-group ```bash [curl] curl https://test-m1.minusonedb.com/init curl https://test-m1.minusonedb.com/init \ -d "username=admin&password=passphrase&bucket=m1-24e27632d234452ba37e60936c0e88eb" ``` ```bash [m1 client] m1 test-m1 init m1 test-m1 init -username admin -password passphrase -bucket m1-24e27632d234452ba37e60936c0e88eb ``` ::: - **rights:** none - **verbs:** GET or POST | Parameter | Type | Required | |-----------|------|----------| | username | String | Yes | | password | String | Yes | | bucket | String | Yes | ### /health Returns the status of all environment components. ::: code-group ```bash [curl] curl https://test-m1.minusonedb.com/health \ -H "m1-auth-token: $myToken" ``` ```bash [m1 client] m1 test-m1 health ``` ::: - **rights:** admin - **verbs:** GET - **parameters:** none ## Rights > An example response when a user does not have sufficient rights: ```json { "message": "User test does not have requested right in set admin", "status": 403 } ``` Users must have one of the rights listed in the service description in order to use the service. Rights are case-sensitive strings. The list below details all MinusOneDB managed environment rights. **admin:** user can manage users, assign rights, and perform typical operational tasks (backup, restore, health checks, etc). **schema:** user can edit and view the environment schema. **get:** user can query data the data lake or data stores. **publish:** user can publish data to the data lake. **sessionQuery:** user can query all session stores. **sessionUpdate:** user can add, update, or delete data from all session stores. You can use [/user/update](https://www.minusonedb.com/docs/environment-api#user-update) to update the rights of a user. The collection of rights is extensible for application-specific uses. -------------------------------------------------------------------------------- > Source: https://www.minusonedb.com/docs/ops-api # Ops API The ops services documented below are hosted at `https://ops.minusonedb.com` and allow you to manage your account(s) and [environments](https://www.minusonedb.com/docs/environment-api). You can interact with all of the service endpoints described below via conventional HTTP clients (curl, postman) or programmatically in your desired language or framework. We offer the [m1 client](https://www.minusonedb.com/docs/m1-client) which simplifies credential management and environment configuration. Services use typical HTTP response codes. **2xx:** The request operation worked as as expected. **401:** The token used for the request was invalid. **403:** The user associated with the token used for the request does not have sufficient [rights](https://www.minusonedb.com/docs/ops-rights) to perform the requested operation. **500:** The request could not be completed. -------------------------------------------------------------------------------- > Source: https://www.minusonedb.com/docs/ops-auth-access # Authentication and Access ### /signup ::: code-group ```bash [curl] curl https://ops.minusonedb.com/signup \ -d "username=youremail@email.com&password=passphrase" ``` ```bash [m1 client] m1 signup ``` ::: | Parameter | Type | Required | |-----------|------|----------| | username | String | Yes | | password | String | Yes | - **Rights:** none - **Verb:** POST Creates a user for you on ops. If successful, you can immediately [authenticate](#auth) using the username and password you supplied. ### /auth ::: code-group ```bash [curl] curl https://ops.minusonedb.com/auth \ -d "username=admin@email.com&password=passphrase" ``` ```bash [m1 client] m1 auth ``` ::: | Parameter | Type | Required | |-----------|------|----------| | username | String | Yes | | password | String | Yes | - **Rights:** none - **Verb:** POST Log in and retrieve a token for service access. ### /user/password ::: code-group ```bash [curl] curl https://ops.minusonedb.com/user/password \ -d "password=newPassphrase" \ -H "m1-auth-token: $myToken" ``` ```bash [m1 client] m1 ops user/password -password newPassphrase ``` ::: | Parameter | Type | Required | |-----------|------|----------| | password | String | Yes | - **Rights:** none - **Verb:** POST Change user password. A correct auth token must be included in the request. -------------------------------------------------------------------------------- > Source: https://www.minusonedb.com/docs/ops-accounts # Accounts ### /account/list ::: code-group ```bash [curl] curl https://ops.minusonedb.com/account/list \ -H "m1-auth-token: $myToken" ``` ```bash [m1 client] m1 ops account/list ``` ::: - **Rights:** none - **Verb:** GET List all accounts the current user has access to. ### /account/user/list ::: code-group ```bash [curl] curl https://ops.minusonedb.com/account/user/list?account=31e1192e-5415-4d85-a45e-b88897c96c1bd \ -H "m1-auth-token: $myToken" ``` ```bash [m1 client] m1 ops account/user/list -account 31e1192e-5415-4d85-a45e-b88897c96c1bd ``` ::: | Parameter | Type | Required | |-----------|------|----------| | account | String | Yes | - **Rights:** none - **Verb:** GET List all users associated with this account. ### /account/user/add ::: code-group ```bash [curl] curl https://ops.minusonedb.com/account/user/add \ -d 'account=31e1192e-5415-4d85-a45e-b88897c96c1b&username=user@email.com&rights=["view"]' \ -H "m1-auth-token: $myToken" ``` ```bash [m1 client] m1 ops account/user/add -account 31e1192e-5415-4d85-a45e-b88897c96c1b -username user@email.com -rights '["view"]' ``` ::: | Parameter | Type | Required | |-----------|------|----------| | account | String | Yes | | username | String | Yes | | rights | Array | No | - **Rights:** admin - **Verb:** POST Add access to an account for the specified user. Rights defaults to no rights if unspecified. ### /account/user/update ::: code-group ```bash [curl] curl https://ops.minusonedb.com/account/user/update \ -d 'account=31e1192e-5415-4d85-a45e-b88897c96c1b&username=user@email.com&rights=["manage","view"]' \ -H "m1-auth-token: $myToken" ``` ```bash [m1 client] m1 ops account/user/update -account 31e1192e-5415-4d85-a45e-b88897c96c1b -username user@email.com \ -rights '["manage", "view"]' ``` ::: | Parameter | Type | Required | |-----------|------|----------| | account | String | Yes | | username | String | Yes | | rights | Array | Yes | - **Rights:** admin - **Verb:** POST Update account rights for the specified user. ### /account/user/remove ::: code-group ```bash [curl] curl https://ops.minusonedb.com/account/user/remove \ -d "account=31e1192e-5415-4d85-a45e-b88897c96c1b&username=user@email.com" \ -H "m1-auth-token: $myToken" ``` ```bash [m1 client] m1 ops account/user/remove -account 31e1192e-5415-4d85-a45e-b88897c96c1b -username user@email.com ``` ::: | Parameter | Type | Required | |-----------|------|----------| | account | String | Yes | | username | String | Yes | - **Rights:** admin - **Verb:** POST Remove all access to account for the specified user. -------------------------------------------------------------------------------- > Source: https://www.minusonedb.com/docs/ops-environments # Environments ### /env/get ::: code-group ```bash [curl] curl https://ops.minusonedb.com/env/get?env=24e27632d234452ba37e60936c0e88eb \ -H "m1-auth-token: $myToken" ``` ```bash [m1 client] m1 ops env/get -env 24e27632d234452ba37e60936c0e88eb ``` ::: | Parameter | Type | Required | |-----------|------|----------| | env | String | Yes | - **Rights:** view - **Verb:** GET Retrieve metadata associated with the specified environment. Note: Anywhere the environment id is passed into, the environment name could be used as well. ### /env/list ::: code-group ```bash [curl] curl https://ops.minusonedb.com/env/list?account=31e1192e-5415-4d85-a45e-b88897c96c1b \ -H "m1-auth-token: $myToken" ``` ```bash [m1 client] m1 ops env/list -account 31e1192e-5415-4d85-a45e-b88897c96c1b ``` ::: | Parameter | Type | Required | |-----------|------|----------| | account | String | Yes | - **Rights:** view - **Verb:** GET List all environments associated with the specified account. ### /env/create ::: code-group ```bash [curl] curl https://ops.minusonedb.com/env/create \ -d "env=newenvironment-m1&servers=2" \ -H "m1-auth-token: $myToken" ``` ```bash [m1 client] m1 ops env/create -env newenvironment-m1 -servers 2 ``` ::: | Parameter | Type | Required | |-----------|------|----------| | account | String | Yes | | name | String | Yes | | servers | int | No | - **Rights:** manage - **Verb:** POST Create a new environment. The name must be unique within the account. Servers defaults to 2 if unspecified. Note: Anywhere the environment id is passed into, the environment name could be used as well. ### /env/rename ::: code-group ```bash [curl] curl https://ops.minusonedb.com/env/rename \ -d "env=myenv-m1&name=mynewenv-m1" \ -H "m1-auth-token: $myToken" ``` ```bash [m1 client] m1 ops env/rename -env myenv-m1 -name mynewenv-m1 ``` ::: | Parameter | Type | Required | |-----------|------|----------| | env | String | Yes | | name | String | Yes | - **Rights:** manage - **Verb:** POST Renames an existing environment. Can be used to move an environment from one account to another but the user must have `manage` permissions for both the source and destination accounts. ### /env/destroy ::: code-group ```bash [curl] curl https://ops.minusonedb.com/env/destroy \ -d "env=24e27632d234452ba37e60936c0e88eb" \ -H "m1-auth-token: $myToken" ``` ```bash [m1 client] m1 ops env/destroy -env 24e27632d234452ba37e60936c0e88eb ``` ::: | Parameter | Type | Required | |-----------|------|----------| | env | String | Yes | - **Rights:** manage - **Verb:** POST Destroy an environment and all associated resources. Note: Anywhere the environment id is passed into, the environment name could be used as well. ### /env/rescale ::: code-group ```bash [curl] curl https://ops.minusonedb.com/env/rescale \ -d "env=24e27632d234452ba37e60936c0e88eb&servers=2&threshold=2" \ -H "m1-auth-token: $myToken" ``` ```bash [m1 client] m1 ops env/rescale -env 24e27632d234452ba37e60936c0e88eb -servers 2 -threshold 2 ``` ::: | Parameter | Type | Required | |-----------|------|----------| | env | String | Yes | | servers | int | No | | threshold | int | No | - **Rights:** manage - **Verb:** POST Change the number of data processing servers for the specified environment. If servers is unspecified, the existing number is used. Threshold defaults to the existing number of servers. Note: Anywhere the environment id is passed into, the environment name could be used as well. ### /env/reboot ::: code-group ```bash [curl] curl https://ops.minusonedb.com/env/reboot \ -d "env=24e27632d234452ba37e60936c0e88eb&group=index" \ -H "m1-auth-token: $myToken" ``` ```bash [m1 client] m1 ops env/reboot -env 24e27632d234452ba37e60936c0e88eb -group index ``` ::: | Parameter | Type | Required | |-----------|------|----------| | env | String | Yes | | group | String | Yes | - **Rights:** manage - **Verb:** POST Reboots all servers in a server group; a server group encompasses the servers associated for a particular data store or session store. Note: Anywhere the environment id is passed into, the environment name could be used as well. ### /env/store/create ::: code-group ```bash [curl] curl https://ops.minusonedb.com/env/store/create \ -d "env=24e27632d234452ba37e60936c0e88eb&store=index&type=standard&shards=1&replicas=2" \ -H "m1-auth-token: $myToken" ``` ```bash [m1 client] m1 ops env/store/create -env 24e27632d234452ba37e60936c0e88eb -store index -type standard -shards 1 -replicas 2 ``` ::: | Parameter | Type | Required | |-----------|------|----------| | env | String | Yes | | store | String | Yes | | type | String | No | | shards | int | No | | replicas | int | No | - **Rights:** manage - **Verb:** POST Create a new data store for the specified environment. Store name must be unique within the environment. Type is `standard` or `boost` (default: standard). Shards default to 1; a single shard can store 450 GB (standard) or 375 GB (boost). Replicas default to 2. Note: Anywhere the environment id is passed into, the environment name could be used as well. ### /env/store/rename ::: code-group ```bash [curl] curl https://ops.minusonedb.com/env/store/rename \ -d "env=24e27632d234452ba37e60936c0e88eb&store=index&name=newStoreName" \ -H "m1-auth-token: $myToken" ``` ```bash [m1 client] m1 ops env/store/rename -env 24e27632d234452ba37e60936c0e88eb -store index -name newStoreName ``` ::: | Parameter | Type | Required | |-----------|------|----------| | env | String | Yes | | store | String | Yes | | name | String | Yes | - **Rights:** manage - **Verb:** POST Change the name of a data store for the specified environment. Note: Anywhere the environment id is passed into, the environment name could be used as well. ### /env/store/destroy ::: code-group ```bash [curl] curl https://ops.minusonedb.com/env/store/destroy \ -d "env=24e27632d234452ba37e60936c0e88eb&store=index" \ -H "m1-auth-token: $myToken" ``` ```bash [m1 client] m1 ops env/store/destroy -env 24e27632d234452ba37e60936c0e88eb -store index ``` ::: | Parameter | Type | Required | |-----------|------|----------| | env | String | Yes | | store | String | Yes | - **Rights:** manage - **Verb:** POST Destroy a data store from the specified environment. Note: Anywhere the environment id is passed into, the environment name could be used as well. ### /env/session/create ::: code-group ```bash [curl] curl https://ops.minusonedb.com/env/session/create \ -d "env=24e27632d234452ba37e60936c0e88eb&store=stash&shards=1&replicas=2" \ -H "m1-auth-token: $myToken" ``` ```bash [m1 client] m1 ops env/session/create -env 24e27632d234452ba37e60936c0e88eb -store stash -shards 1 -replicas 2 ``` ::: | Parameter | Type | Required | |-----------|------|----------| | env | String | Yes | | store | String | Yes | | shards | int | No | | replicas | int | No | - **Rights:** manage - **Verb:** POST Create a new session store for the specified environment. Store name must be unique within the environment. Shards default to 1; a single shard can store approximately 450,000 session objects and support approximately 45,000 updates/second. Replicas default to 2. Note: Anywhere the environment id is passed into, the environment name could be used as well. ### /env/session/rename ::: code-group ```bash [curl] curl https://ops.minusonedb.com/env/session/rename \ -d "env=24e27632d234452ba37e60936c0e88eb&store=stash&name=newSessionStoreName" \ -H "m1-auth-token: $myToken" ``` ```bash [m1 client] m1 ops env/session/rename -env 24e27632d234452ba37e60936c0e88eb -store stash -name newSessionStoreName ``` ::: | Parameter | Type | Required | |-----------|------|----------| | env | String | Yes | | store | String | Yes | | name | String | Yes | - **Rights:** manage - **Verb:** POST Change the name for a session store for the specified environment. Note: Anywhere the environment id is passed into, the environment name could be used as well. ### /env/session/destroy ::: code-group ```bash [curl] curl https://ops.minusonedb.com/env/session/destroy \ -d "env=24e27632d234452ba37e60936c0e88eb&store=stash" \ -H "m1-auth-token: $myToken" ``` ```bash [m1 client] m1 ops env/session/destroy -env 24e27632d234452ba37e60936c0e88eb -store stash ``` ::: | Parameter | Type | Required | |-----------|------|----------| | env | String | Yes | | store | String | Yes | - **Rights:** manage - **Verb:** POST Destroy a session store from the specified environment. Note: Anywhere the environment id is passed into, the environment name could be used as well. ### /env/bucket/register ::: code-group ```bash [curl] curl https://ops.minusonedb.com/env/bucket/register \ -d "env=24e27632d234452ba37e60936c0e88eb&bucket=myBucket" \ -H "m1-auth-token: $myToken" ``` ```bash [m1 client] m1 ops env/bucket/register -env 24e27632d234452ba37e60936c0e88eb -bucket myBucket ``` ::: | Parameter | Type | Required | |-----------|------|----------| | env | String | Yes | | bucket | String | Yes | - **Rights:** manage - **Verb:** POST Register a new bucket for the specified environment. Note: Anywhere the environment id is passed into, the environment name could be used as well. ### /env/bucket/deregister ::: code-group ```bash [curl] curl https://ops.minusonedb.com/env/bucket/deregister \ -d "env=24e27632d234452ba37e60936c0e88eb&bucket=myBucket" \ -H "m1-auth-token: $myToken" ``` ```bash [m1 client] m1 ops env/bucket/deregister -env 24e27632d234452ba37e60936c0e88eb -bucket myBucket ``` ::: | Parameter | Type | Required | |-----------|------|----------| | env | String | Yes | | bucket | String | Yes | - **Rights:** manage - **Verb:** POST Deregister an existing bucket from the specified environment. Note: Anywhere the environment id is passed into, the environment name could be used as well. ### /env/bucket/grant ::: code-group ```bash [curl] curl https://ops.minusonedb.com/env/bucket/grant \ -d "env=24e27632d234452ba37e60936c0e88eb&arn=arn:aws:iam::123456789123456789:role/read-role" \ -H "m1-auth-token: $myToken" ``` ```bash [m1 client] m1 ops env/bucket/grant -env 24e27632d234452ba37e60936c0e88eb -arn arn:aws:iam::123456789123456789:role/read-role ``` ::: | Parameter | Type | Required | |-----------|------|----------| | env | String | Yes | | arn | String | Yes | - **Rights:** manage - **Verb:** POST Grant read access to the environment bucket to a user's role external to the m1 AWS account. Try [env/get](https://www.minusonedb.com/docs/ops-environments#env-get) to see what read roles have been granted. Note: Anywhere the environment id is passed into, the environment name could be used as well. ### /env/bucket/revoke ::: code-group ```bash [curl] curl https://ops.minusonedb.com/env/bucket/revoke \ -d "env=24e27632d234452ba37e60936c0e88eb&arn=arn:aws:iam::123456789123456789:role/read-role" \ -H "m1-auth-token: $myToken" ``` ```bash [m1 client] m1 ops env/bucket/revoke -env 24e27632d234452ba37e60936c0e88eb -arn arn:aws:iam::123456789123456789:role/read-role ``` ::: | Parameter | Type | Required | |-----------|------|----------| | env | String | Yes | | arn | String | Yes | - **Rights:** manage - **Verb:** POST Revoke read access to the environment bucket to a user's role external to the m1 AWS account. Try [env/get](https://www.minusonedb.com/docs/ops-environments#env-get) to see what read roles have been granted. Note: Anywhere the environment id is passed into, the environment name could be used as well. ### /env/outbound ::: code-group ```bash [curl] curl https://ops.minusonedb.com/env/outbound \ -d "env=test-m1&enable=true" \ -H "m1-auth-token: $myToken" ``` ```bash [m1 client] m1 ops env/outbound -env test-m1 -enable true ``` ::: | Parameter | Type | Required | |-----------|------|----------| | env | String | Yes | | enable | Boolean | Yes | - **Rights:** manage - **Verb:** POST Enables or disables outbound internet connectivity for an environment's data processing servers. This is disabled by default but must be enabled to access data in GCS buckets. -------------------------------------------------------------------------------- > Source: https://www.minusonedb.com/docs/ops-rights # Rights Rights are case-sensitive strings associated with a specific account; a user may have access to many (or no) accounts with varying levels of access. The collection of rights is extensible for application-specific uses. The list below details all ops.minusonedb.com rights. **owner:** user can delete the account or change payment information. **admin:** user can manage users and assign rights. **view:** user can view all account assets including environment names and metadata. **manage:** user can manage environment assets including deploying or undeploying infrastructure as well as creating and destroying environments. -------------------------------------------------------------------------------- > Source: https://www.minusonedb.com/docs/troubleshooting # Troubleshooting ## Common API Errors ### 401 Errors > Sample 401 Response: ``` {\ "message" : "Invalid Token",\ "status" : 401\ }\ ``` Most [ops](https://www.minusonedb.com/docs/ops-api#auth) and [environment](https://www.minusonedb.com/docs/environment-api#auth) services require an authentication token. If a particular request fails with a 401 error, doublecheck the following: - Ensure that you are using a token generated from the environment you are attempting to access (you will need different tokens to access `https://ops.minusonedb.com` and each of your environments). - Ensure that you are including the m1-auth-token HTTP header in your request. The api reference sections in this documentation show a number of `curl` examples that illustrate how this is done. - By default, tokens can only be used from the IP address from where they were originally generated. If you are using a token generated from a different machine, you will need to create a new token on that machine. - Tokens will expire after a user configurable [time](https://www.minusonedb.com/docs/env-system-config#system-token-expire-ms). If your token has expired, you will need to generate a new token. [/token/data](https://www.minusonedb.com/docs/env-users-tokens#token-data) can provide some additional diagnostic information that may be helpful in understanding why a token is not working. Consider using the [m1 client](https://www.minusonedb.com/docs/m1-client) to manage the token life cycle for you. ## Java Setup > Successful java -version output: you already have a JRE installed ``` openjdk version "16.0.1" 2021-04-20\ OpenJDK Runtime Environment (build 16.0.1+9-Ubuntu-120.04)\ OpenJDK 64-Bit Server VM (build 16.0.1+9-Ubuntu-120.04, mixed mode, sharing)\ ``` > Unsuccessful java -version output: you need to install a JRE. ``` java: command not found\ ``` The [m1 client](https://www.minusonedb.com/docs/m1-client) requires a Java Runtime Environment (JRE) version 1.8 or higher. You may already have a java runtime installed on your computer. To check, run the following: `java -version` The exact output may vary (some samples are below) but if it looks like you do not have a JRE installed, download and install the latest for your platform from [https://adoptium.net/](https://adoptium.net/). ## Python Setup To run any MinusOneDB samples that require python, you will need to install python3 and some 3rd party python packages. _Step 1:_ [Install python3](https://realpython.com/installing-python) _Step 2:_ Install 3rd party libraries used in MinusOneDB samples. **Ubuntu** `sudo apt-get install python3-boto3 python3-pebble` **Windows & Mac** `pip install boto3 pebble requests` If you are using a python package manager other than `pip`, refer to its documentation to install the `boto3` and `pebble` packages on your system. -------------------------------------------------------------------------------- > Source: https://www.minusonedb.com/docs/release-notes # Release Notes
Timestamp Component Description
2026-03-30 10:38 ops
  • No user facing changes
2026-03-30 10:38 svc
  • Improve error messages when there is a failure to connect to data or session store
  • Improve error messages when there is a failure to connect to internal db
  • Add % progress to geo loading status in /health; percentCompleted and count are always available as keys in the geo status
2026-03-30 10:38 session
  • No user facing changes
2026-03-30 10:38 store
  • No user facing changes
2026-03-23 17:29 ops
  • No user facing changes
2026-03-23 17:29 svc
  • No user facing changes
2026-03-23 17:29 session
  • No user facing changes
2026-03-23 17:29 store
  • No user facing changes
2026-03-17 14:51 ops
  • Add /env/outbound endpoint with param enable (true/false); this either enables or disables outbound access to the public internet for environment servers. If you are loading data from a gcs bucket you must enable outbound connectivity
  • env/get now returns an additional outbound attribute indicating whether the environment has outbound access to the public internet
2026-03-17 14:51 svc
  • Support /publish for files in gcs buckets; use gs://yourgcsbucket/file.csv
  • the s3file param to /publish is now deprecated in favor of file; it will continue to be supported but clients should migrate to file, as it may be removed in a future release
2026-03-15 14:49 ops
  • No user facing changes
2026-03-15 14:49 svc
  • No user facing changes
2026-03-15 14:49 session
  • No user facing changes
2026-03-15 14:49 store
  • No user facing changes
2026-03-11 11:55 svc
  • Fix regression introduced in 2026-03-08 release where enabling geo would cause an infinite loop of loading geo data
2026-03-08 20:33 ops
  • Address CVE-2026-2006
  • Address CVE-2026-2007
2026-03-08 20:33 svc
  • Address CVE-2026-2006
  • Address CVE-2026-2007
2026-03-08 20:33 session
  • No user facing changes
2026-03-08 20:33 store
  • No user facing changes
2026-02-27 16:46 ops
  • No user facing changes
2026-02-27 16:46 svc
  • Add store/backup/rename endpoint: rename a backup in an environment
  • Fix theoretical sql injection issue with name param of store/backup/create
2026-02-27 16:46 session
  • No user facing changes
2026-02-27 16:46 store
  • No user facing changes
2026-02-20 15:04 ops
  • No user facing changes
2026-02-20 15:04 svc
  • Fix behavior of /query when sorting by a field that is unpopulated; now documents with null values will always sort to the end. This will be a breaking change if the previous buggy behavior was relied upon Note that to see this change on existing store, you must readd all properties via /schema/add after running /schema/wipe
2026-02-20 15:04 session
  • No user facing changes
2026-02-20 15:04 store
  • No user facing changes
2026-02-13 13:44 ops
  • Address CVE-2026-2003
  • Address CVE-2026-2004
  • Address CVE-2026-2005
  • Address CVE-2026-2006
  • Address CVE-2026-2007
2026-02-13 13:44 svc
  • Address CVE-2026-2003
  • Address CVE-2026-2004
  • Address CVE-2026-2005
  • Address CVE-2026-2006
  • Address CVE-2026-2007
2026-02-13 13:44 session
  • No user facing changes
2026-02-13 13:44 store
  • No user facing changes
2026-02-06 17:08 ops
  • Address CVE-2025-43368
  • Address CVE-2025-7425
  • Address CVE-2026-21945
  • Address CVE-2026-21932
  • Address CVE-2026-21933
  • Address CVE-2025-6021
  • Address CVE-2025-12183
  • Address CVE-2026-21925
  • Address CVE-2025-6052
  • Address CVE-2026-21947
  • Address CVE-2025-47219
2026-02-06 17:08 svc
  • Any successful call to /schema/add now will always sync schemas of underlying data stores; this allows one to run m1 env-yours schema/add -properties '[]' after adding a store to sync its schema
  • Address CVE-2025-43368
  • Address CVE-2025-7425
  • Address CVE-2026-21945
  • Address CVE-2026-21932
  • Address CVE-2026-21933
  • Address CVE-2025-6021
  • Address CVE-2025-12183
  • Address CVE-2026-21925
  • Address CVE-2025-6052
  • Address CVE-2026-21947
  • Address CVE-2025-47219
2026-02-06 17:08 session
  • Address CVE-2025-43368
  • Address CVE-2025-7425
  • Address CVE-2026-21945
  • Address CVE-2026-21932
  • Address CVE-2026-21933
  • Address CVE-2025-6021
  • Address CVE-2025-12183
  • Address CVE-2026-21925
  • Address CVE-2025-6052
  • Address CVE-2026-21947
  • Address CVE-2025-47219
2026-02-06 17:08 store
  • No user facing changes
2026-01-30 11:54 ops
  • No user facing changes
2026-01-30 11:54 svc
  • Fix bug where store/backup/create would not properly set status of backup to "failed" if an error occured during the backup; the status would have remained in the "working" state indefinitely.
2026-01-30 11:54 session
  • No user facing changes
2026-01-30 11:54 store
  • No user facing changes
2026-01-23 12:06 ops
  • No user facing changes
2026-01-23 12:06 svc
  • No user facing changes
2026-01-23 12:06 store
  • No user facing changes
2026-01-23 12:06 session
  • No user facing changes
2026-01-16 15:26 ops
  • No user facing changes
2026-01-16 15:26 svc
  • No user facing changes
2026-01-16 15:26 session
  • No user facing changes
2026-01-16 15:26 store
  • No user facing changes
2026-01-09 16:10 ops
  • No user facing changes
2026-01-09 16:10 svc
  • No user facing changes
2026-01-09 16:10 session
  • No user facing changes
2026-01-09 16:10 store
  • No user facing changes
2026-01-02 12:25 ops
  • No user facing changes
2026-01-02 12:25 svc
  • No user facing changes
2026-01-02 12:25 session
  • No user facing changes
2026-01-02 12:25 store
  • No user facing changes
2025-12-26 16:29 ops
  • No user facing changes
2025-12-26 16:29 svc
  • No user facing changes
2025-12-26 16:29 session
  • No user facing changes
2025-12-26 16:29 store
  • No user facing changes
2025-12-19 17:14 ops
  • No user facing changes
2025-12-19 17:14 svc
  • No user facing changes
2025-12-19 17:14 store
  • No user facing changes
2025-12-19 17:14 session
  • No user facing changes
2025-12-12 15:46 ops
  • No user facing changes
2025-12-12 15:46 svc
  • No user facing changes
2025-12-12 15:46 session
  • No user facing changes
2025-12-12 15:46 store
  • No user facing changes
2025-12-05 14:39 ops
  • No user facing changes
2025-12-05 14:39 svc
  • No user facing changes
2025-12-05 14:39 session
  • No user facing changes
2025-12-05 14:39 store
  • No user facing changes
2025-11-28 16:57 ops
  • No user facing changes
2025-11-28 16:57 svc
  • No user facing changes
2025-11-28 16:57 session
  • No user facing changes
2025-11-28 16:57 store
  • No user facing changes
2025-11-21 16:30 ops
  • No user facing changes
2025-11-21 16:30 svc
  • No user facing changes
2025-11-21 16:30 session
  • No user facing changes
2025-11-21 16:30 store
  • No user facing changes
2025-11-14 14:19 ops
  • Address CVE-2025-12817
  • Address CVE-2025-12818
2025-11-14 14:19 svc
  • Address CVE-2025-12817
  • Address CVE-2025-12818
2025-11-14 14:19 session
  • No user facing changes
2025-11-14 14:19 store
  • No user facing changes
2025-11-07 14:53 ops
  • No user facing changes
2025-11-07 14:53 svc
  • No user facing changes
2025-11-07 14:53 session
  • No user facing changes
2025-11-07 14:53 store
  • No user facing changes
2025-10-31 20:54 svc
  • Add store/backup/drop endpoint: deletes a backup from an environment
2025-10-31 19:55 ops
  • No user facing changes
2025-10-31 19:55 svc
  • No user facing changes
2025-10-31 19:55 session
  • No user facing changes
2025-10-31 19:55 store
  • No user facing changes
2025-10-2416:10 ops
  • No user facing changes
2025-10-2416:10 svc
  • No user facing changes
2025-10-2416:10 session
  • No user facing changes
2025-10-2416:10 store
  • No user facing changes
2025-10-17 21:59 ops
  • No user facing changes
2025-10-17 21:59 svc
  • No user facing changes
2025-10-17 21:59 session
  • No user facing changes
2025-10-17 21:59 store
  • No user facing changes
2025-10-10 14:20 ops
  • No user facing changes
2025-10-10 14:20 svc
  • No user facing changes
2025-10-10 14:20 session
  • No user facing changes
2025-10-10 14:20 store
  • No user facing changes
2025-10-03 14:11 ops
  • No user facing changes
2025-10-03 14:11 svc
  • No user facing changes
2025-10-03 14:11 session
  • No user facing changes
2025-10-03 14:11 store
  • No user facing changes
2025-09-26 14:40 ops
  • No user facing changes
2025-09-26 14:40 svc
  • Breaking change: /query endpoint can be used to request documents or facets but not both in 1 query
  • Dramatically improve performance of certain types of queries in multishard environments; previously the more shards one had, responses with very large /query outputs would become linearly slower with the number of shards.
2025-09-26 14:40 session
  • No user facing changes
2025-09-26 14:40 store
  • No user facing changes
2025-09-19 14:44 ops
  • No user facing changes
2025-09-19 14:44 svc
  • No user facing changes
2025-09-19 14:44 session
  • No user facing changes
2025-09-19 14:44 store
  • No user facing changes
2025-09-12 20:57 ops
  • No user facing changes
2025-09-12 20:57 svc
  • No user facing changes
2025-09-12 20:57 session
  • No user facing changes
2025-09-12 20:57 store
  • No user facing changes
2025-09-05 22:31 ops
  • No user facing changes
2025-09-05 22:31 svc
  • Fix bug where in an environment where publish-permissive is set to false and a transform rule set is used, a /publish of data will incorrectly "succeed" even when there is a mismatch between schema and data (the output data will simply be null, like when publish-permissive is set to true, rather than throwing an error as it should in this mode).
2025-09-05 22:31 session
  • No user facing changes
2025-09-05 22:31 store
  • No user facing changes
2025-08-29 17:02 ops
  • No user facing changes
2025-08-29 17:02 svc
  • No user facing changes
2025-08-29 17:02 session
  • No user facing changes
2025-08-29 17:02 store
  • No user facing changes
2025-08-22 19:15 ops
  • Fix http status of various environment modifying related operations; service calls that should return a 400 (for example, referencing a non existent store) would return a 500 rather than 400
2025-08-22 19:15 svc
  • Make proper error message and http status code if one calls env/store/backup with an invalid backup id
2025-08-22 19:15 session
  • No user facing changes
2025-08-22 19:15 store
  • No user facing changes
2025-08-15 18:15 ops
  • Address CVE-2025-8713
  • Address CVE-2025-8714
  • Address CVE-2025-8715
2025-08-15 18:15 svc
  • Address CVE-2025-8713
  • Address CVE-2025-8714
  • Address CVE-2025-8715
2025-08-15 18:15 session
  • No user facing changes
2025-08-15 18:15 store
  • No user facing changes
2025-08-08 17:18vd ops
  • Fix bug where env/reboot would return a 500 with ugly error message when trying to reboot a non existent store; now returns a proper 400 with "No such store" error message
  • Remove any 409 or 404 error codes returned by ops methods in favor of a 400; this is technically a breaking change but the 400 status code in this case is more of an industry standard
2025-08-08 17:18vd svc
  • No user facing changes
2025-08-08 17:18vd session
  • No user facing changes
2025-08-08 17:18vd store
  • No user facing changes
2025-08-01 12:57 ops
  • No user facing changes
2025-08-01 12:57 svc
  • No user facing changes
2025-08-01 12:57 session
  • No user facing changes
2025-08-01 12:57 store
  • No user facing changes
2025-07-27 20:39 reports
  • Fix security issue where html/javascript content stored in an m1db environment could potentially be executed; this would typically result in errors but would generally be blocked by Content-Security-Policy settings and would not result in any sort of data leakage.
2025-07-25 20:27 ops
  • No user facing changes
2025-07-25 20:27 svc
  • No user facing changes
2025-07-25 20:27 session
  • No user facing changes
2025-07-25 20:27 store
  • No user facing changes
2025-07-18 13:24 ops
  • No user facing changes
2025-07-18 13:24 svc
  • No user facing changes
2025-07-18 13:24 session
  • No user facing changes
2025-07-18 13:24 store
  • No user facing changes
2025-07-11 13:16 ops
  • No user facing changes
2025-07-11 13:16 svc
  • No user facing changes
2025-07-11 13:16 session
  • No user facing changes
2025-07-11 13:16 store
  • No user facing changes
2025-07-06 18:07 ops
  • No user facing changes
2025-07-06 18:07 svc
  • No user facing changes
2025-07-06 18:07 session
  • No user facing changes
2025-07-06 18:07 store
  • No user facing changes
2025-06-28 14:09 ops
  • Fix bug where env/store/deploy would fail if trying to rename a store with more than 1 shard; one would get the error "Unable to rename store. java.util.ConcurrentModificationException",
2025-06-27 21:08 ops
  • Fix regression introduced in 2025-03-14 release that broke env/store/rename and /env/session/rename endpoints; any use of these endpoints would return an error like "Unable to rename store. 400 BAD_REQUEST \"No null `index` exists."
2025-06-27 15:50 ops
  • No user facing changes
2025-06-27 15:50 svc
  • No user facing changes
2025-06-27 15:50 session
  • No user facing changes
2025-06-27 15:50 store
  • No user facing changes
2025-06-20 13:05 ops
  • No user facing changes
2025-06-20 13:05 svc
  • No user facing changes
2025-06-20 13:05 session
  • No user facing changes
2025-06-20 13:05 store
  • No user facing changes
2025-06-13 18:24 ops
  • No user facing changes
2025-06-13 18:24 svc
  • No user facing changes
2025-06-13 18:24 session
  • No user facing changes
2025-06-13 18:24 store
  • No user facing changes
2025-06-06 14:29 ops
  • No user facing changes
2025-06-06 14:29 svc
  • No user facing changes
2025-06-06 14:29 session
  • No user facing changes
2025-06-06 14:29 store
  • No user facing changes
2025-05-30 13:49 ops
  • No user facing changes
2025-05-30 13:49 svc
  • No user facing changes
2025-05-30 13:49 session
  • No user facing changes
2025-05-30 13:49 store
  • No user facing changes
2025-05-23 11:38 ops
  • No user facing changes
2025-05-23 11:38 svc
  • No user facing changes
2025-05-23 11:38 session
  • No user facing changes
2025-05-23 11:38 store
  • No user facing changes
2025-05-16 9:50 ops
  • No user facing changes
2025-05-16 9:50 svc
  • No user facing changes
2025-05-16 9:50 session
  • No user facing changes
2025-05-16 9:50 store
  • No user facing changes
2025-05-09 12:30 ops
  • Address CVE-2025-4207
2025-05-09 12:30 svc
  • Address CVE-2025-4207
2025-05-09 12:30 session
  • No user facing changes
2025-05-09 12:30 store
  • No user facing changes
2025-05-02 10:35 ops
  • No user facing changes
2025-05-02 10:35 svc
  • No user facing changes
2025-05-02 10:35 session
  • No user facing changes
2025-05-02 10:35 store
  • No user facing changes
2025-04-26 14:29 ops
  • No user facing changes
2025-04-26 14:29 svc
  • No user facing changes
2025-04-26 14:29 session
  • No user facing changes
2025-04-26 14:29 store
  • No user facing changes
2025-04-18 9:36 ops
  • Address CVE-2025-23083
  • Address CVE-2024-54534
  • Address CVE-2024-47606
  • Address CVE-2025-21587
  • Address CVE-2025-30698
  • Address CVE-2025-30691
2025-04-18 9:36 svc
  • Address CVE-2025-23083
  • Address CVE-2024-54534
  • Address CVE-2024-47606
  • Address CVE-2025-21587
  • Address CVE-2025-30698
  • Address CVE-2025-30691
2025-04-18 9:36 session
  • Address CVE-2025-23083
  • Address CVE-2024-54534
  • Address CVE-2024-47606
  • Address CVE-2025-21587
  • Address CVE-2025-30698
  • Address CVE-2025-30691
2025-04-18 9:36 store
  • Address CVE-2025-23083
  • Address CVE-2024-54534
  • Address CVE-2024-47606
  • Address CVE-2025-21587
  • Address CVE-2025-30698
  • Address CVE-2025-30691
2025-04-11 11:16 ops
  • No user facing changes
2025-04-11 11:16 svc
  • No user facing changes
2025-04-11 11:16 session
  • No user facing changes
2025-04-11 11:16 store
  • No user facing changes
2025-04-04 18:26 ops
  • No user facing changes
2025-04-04 18:26 svc
  • No user facing changes
2025-04-04 18:26 session
  • No user facing changes
2025-04-04 18:26 store
  • No user facing changes
2025-03-28 10:26 ops
  • No user facing changes
2025-03-28 10:26 svc
  • No user facing changes
2025-03-28 10:26 session
  • No user facing changes
2025-03-28 10:26 store
  • No user facing changes
2025-03-21 10:17 ops
  • No user facing changes
2025-03-21 10:17 svc
  • No user facing changes
2025-03-21 10:17 session
  • No user facing changes
2025-03-21 10:17 store
  • No user facing changes
2025-03-14 11:01 ops
  • system properties are now gettable and settable via /system/properties rather than /system/. All properties may still be retrieved via a call to /system but this may be deprecated in the future.
  • A number of changes to authentication:
  • Authentication tokens are no longer stateful and may not be listed or invalidated individually.
  • Given that tokens are no longer stateful, we recommend a shorter ttl on tokens. The default for newly created environments is now 10 minutes; you can get the same behavior on your existing environment by running something like "m1 your-env system/properties/token-expire-ms -value 600000"
  • Adds support for JWTs to be used for authentication (authn) and authorization (authz) by peer applications.
  • JWTs follow the standard spec in all regards with the exception of using longer keylengths for JWT signing.
  • Remote applications may validate keys and retrieve claims either by service call or by validating signature remotely using an available public key.
  • JWTs are not encrypted by default but may be encrypted by provding a secret to be used to encrypt using symmetric encryption via the encryption-secret property in /system/properties.
  • Fix status code for authentication failures -- was previously 500 but should be 401
2025-03-14 11:01 svc
  • A number of changes to authentication:
  • Authentication tokens are no longer stateful and may not be listed or invalidated individually.
  • Given that tokens are no longer stateful, we recommend a shorter ttl on tokens. The default for newly created environments is now 10 minutes; you can get the same behavior on your existing environment by running something like "m1 your-env system/properties/token-expire-ms -value 600000"
  • Adds support for JWTs to be used for authentication (authn) and authorization (authz) by peer applications.
  • JWTs follow the standard spec in all regards with the exception of using longer keylengths for JWT signing.
  • Remote applications may validate keys and retrieve claims either by service call or by validating signature remotely using an available public key.
  • JWTs are not encrypted by default but may be encrypted by provding a secret to be used to encrypt using symmetric encryption via the encryption-secret property in /system/properties.
  • Fix status code for authentication failures -- was previously 500 but should be 401
2025-03-14 11:01 session
  • No user facing changes
2025-03-14 11:01 store
  • No user facing changes
2025-03-07 11:16 ops
  • Improve error message in the event that an env/rescale call fails
2025-03-07 11:16 svc
  • No user facing changes
2025-03-07 11:16 session
  • No user facing changes
2025-03-07 11:16 store
  • No user facing changes
2025-02-28 10:06 ops
  • No user facing changes
2025-02-28 10:06 svc
  • No user facing changes
2025-02-28 10:06 session
  • No user facing changes
2025-02-28 10:06 store
  • No user facing changes
2025-02-21 13:15 ops
  • Address CVE-2004-56337
  • Address CVE-2025-1094
2025-02-21 13:15 svc
  • Address CVE-2004-56337
  • Address CVE-2025-1094
2025-02-21 13:15 session
  • Address CVE-2004-56337
2025-02-21 13:15 store
  • No user facing changes
2025-02-14 10:56 ops
  • No user facing changes
2025-02-14 10:56 svc
  • No user facing changes
2025-02-14 10:56 session
  • No user facing changes
2025-02-14 10:56 store
  • No user facing changes
2025-02-07 11:06 ops
  • No user facing changes
2025-02-07 11:06 svc
  • No user facing changes
2025-02-07 11:06 session
  • No user facing changes
2025-02-07 11:06 store
  • No user facing changes
2025-01-31 15:29 ops
  • Address CVE-2025-0509
  • Address CVE-2025-21502
2025-01-31 15:29 svc
  • Address CVE-2025-0509
  • Address CVE-2025-21502
2025-01-31 15:29 session
  • Address CVE-2025-0509
  • Address CVE-2025-21502
2025-01-31 15:29 store
  • Address CVE-2025-0509
  • Address CVE-2025-21502
2025-01-24 9:52 ops
  • No user facing changes
2025-01-24 9:52 svc
  • No user facing changes
2025-01-24 9:52 session
  • No user facing changes
2025-01-24 9:52 store
  • No user facing changes
2025-01-17 14:36 ops
  • No user facing changes
2025-01-17 14:36 svc
  • No user facing changes
2025-01-17 14:36 session
  • No user facing changes
2025-01-17 14:36 store
  • No user facing changes
2025-01-10 17:11 ops
  • No user facing changes
2025-01-10 17:11 svc
  • No user facing changes
2025-01-10 17:11 session
  • No user facing changes
2025-01-10 17:11 store
  • No user facing changes
2025-01-03 14:36 ops
  • No user facing changes
2025-01-03 14:36 svc
  • No user facing changes
2025-01-03 14:36 session
  • No user facing changes
2025-01-03 14:36 store
  • No user facing changes
2024-12-27 16:30 ops
  • No user facing changes
2024-12-27 16:30 svc
  • No user facing changes
2024-12-27 16:30 session
  • No user facing changes
2024-12-27 16:30 store
  • No user facing changes
2024-12-20 11:06 ops
  • No user facing changes
2024-12-20 11:06 svc
  • No user facing changes
2024-12-20 11:06 session
  • No user facing changes
2024-12-20 11:06 store
  • No user facing changes
2024-12-13 14:07 ops
  • No user facing changes
2024-12-13 14:07 svc
  • No user facing changes
2024-12-13 14:07 session
  • No user facing changes
2024-12-13 14:07 store
  • No user facing changes
2024-12-06 17:14 ops
  • No user facing changes
2024-12-06 17:14 svc
  • No user facing changes
2024-12-06 17:14 session
  • No user facing changes
2024-12-06 17:14 store
  • No user facing changes
2024-11-29 10:27 ops
  • No user facing changes
2024-11-29 10:27 svc
  • No user facing changes
2024-11-29 10:27 session
  • No user facing changes
2024-11-29 10:27 store
  • No user facing changes
2024-11-22 12:07 ops
  • No user facing changes
2024-11-22 12:07 svc
  • No user facing changes
2024-11-22 12:07 session
  • No user facing changes
2024-11-22 12:07 store
  • No user facing changes
2024-11-15 14:21 ops
  • Address CVE-2024-10976
  • Address CVE-2024-10977
  • Address CVE-2024-10978
  • Address CVE-2024-10979
2024-11-15 14:21 svc
  • Address CVE-2024-10976
  • Address CVE-2024-10977
  • Address CVE-2024-10978
  • Address CVE-2024-10979
2024-11-15 14:21 session
  • No user facing changes
2024-11-15 14:21 store
  • No user facing changes
2024-11-08 15:47 ops
  • No user facing changes
2024-11-08 15:47 svc
  • No user facing changes
2024-11-08 15:47 session
  • No user facing changes
2024-11-08 15:47 store
  • No user facing changes
2024-11-02 18:29 ops
  • No user facing changes
2024-11-02 18:29 svc
  • No user facing changes
2024-11-02 18:29 session
  • No user facing changes
2024-11-02 18:29 store
  • No user facing changes
2024-10-25 10:38 ops
  • Address CVE-2024-21235
  • Address CVE-2024-21208
  • Address CVE-2024-21210
  • Address CVE-2024-21217
  • Address CVE-2023-42950
  • Address CVE-2024-25062
2024-10-25 10:38 svc
  • Address CVE-2024-21235
  • Address CVE-2024-21208
  • Address CVE-2024-21210
  • Address CVE-2024-21217
  • Address CVE-2023-42950
  • Address CVE-2024-25062
2024-10-25 10:38 session
  • Address CVE-2024-21235
  • Address CVE-2024-21208
  • Address CVE-2024-21210
  • Address CVE-2024-21217
  • Address CVE-2023-42950
  • Address CVE-2024-25062
2024-10-25 10:38 store
  • Address CVE-2024-21235
  • Address CVE-2024-21208
  • Address CVE-2024-21210
  • Address CVE-2024-21217
  • Address CVE-2023-42950
  • Address CVE-2024-25062
2024-10-23 22:06 ops
  • Add new /env/bucket/grant and /env/bucket/revoke endpoints; These let one grant read rights to an environment's data bucket to an AWS role in one's one account required params: env = environment, arn = AWS entity that will be granted read rights
  • /env/get will have a new key "bucketGrantees" describing roles that have been granted access to the data bucket
2024-10-18 14:59 ops
  • No user facing changes
2024-10-18 14:59 svc
  • No user facing changes
2024-10-18 14:59 session
  • No user facing changes
2024-10-18 14:59 store
  • No user facing changes
2024-10-11 17:53 ops
  • No user facing changes
2024-10-11 17:53 svc
  • No user facing changes
2024-10-11 17:53 session
  • No user facing changes
2024-10-11 17:53 store
  • No user facing changes
2024-10-04 09:57 ops
  • No user facing changes
2024-10-04 09:57 svc
  • No user facing changes
2024-10-04 09:57 session
  • No user facing changes
2024-10-04 09:57 store
  • No user facing changes
2024-09-27 13:18 ops
  • No user facing changes
2024-09-27 13:18 svc
  • No user facing changes
2024-09-27 13:18 session
  • No user facing changes
2024-09-27 13:18 store
  • No user facing changes
2024-09-20 10:48 ops
  • No user facing changes
2024-09-20 10:48 svc
  • No user facing changes
2024-09-20 10:48 session
  • No user facing changes
2024-09-20 10:48 store
  • No user facing changes
2024-09-13 11:34 ops
  • No user facing changes
2024-09-13 11:34 svc
  • No user facing changes
2024-09-13 11:34 session
  • No user facing changes
2024-09-13 11:34 store
  • No user facing changes
2024-09-06 10:23 ops
  • No user facing changes
2024-09-06 10:23 svc
  • No user facing changes
2024-09-06 10:23 session
  • No user facing changes
2024-09-06 10:23 store
  • No user facing changes
2024-08-30 18:01 ops
  • No user facing changes
2024-08-30 18:01 svc
  • No user facing changes
2024-08-30 18:01 session
  • No user facing changes
2024-08-30 18:01 store
  • Fix bug that could cause some long running queries to fail with an "Unknown error while returning streamed response" message.
    This would typically only occur for stores with more than 1 shard.
    Older stores should be backed up and restored to a new store in order to take advantage of this fix.
2024-08-23 20:25 ops
  • No user facing changes
2024-08-23 20:25 svc
  • No user facing changes
2024-08-23 20:25 session
  • No user facing changes
2024-08-23 20:25 store
  • No user facing changes
2024-08-16 15:23 ops
  • No user facing changes
2024-08-16 15:23 svc
  • No user facing changes
2024-08-16 15:23 session
  • No user facing changes
2024-08-16 15:23 store
  • No user facing changes
2024-08-09 14:15 ops
  • Address CVE-2024-7348
  • Address CVE-2023-5869
2024-08-09 14:15 svc
  • Address CVE-2024-7348
  • Address CVE-2023-5869
2024-08-09 14:15 session
  • No user facing changes
2024-08-09 14:15 store
  • No user facing changes
2024-08-02 16:08 ops
  • No user facing changes
2024-08-02 16:08 svc
  • No user facing changes
2024-08-02 16:08 session
  • No user facing changes
2024-08-02 16:08 store
  • No user facing changes
2024-07-26 11:52 ops
  • No user facing changes
2024-07-26 11:52 svc
  • No user facing changes
2024-07-26 11:52 session
  • No user facing changes
2024-07-26 11:52 store
  • No user facing changes
2024-07-19 12:59 ops
  • Address CVE-2024-27983
  • Address CVE-2024-21147
  • Address CVE-2024-21145
  • Address CVE-2024-21140
  • Address CVE-2024-21144
  • Address CVE-2024-21131
  • Address CVE-2024-21138
  • Address CVE-2024-27980
  • Address CVE-2024-27982
2024-07-19 12:59 svc
  • Fix bug where a 500 (rather than a 400) http status code was returned for invalid calls to /schema/add (missing name, invalid type, etc.)
  • Address CVE-2024-27983
  • Address CVE-2024-21147
  • Address CVE-2024-21145
  • Address CVE-2024-21140
  • Address CVE-2024-21144
  • Address CVE-2024-21131
  • Address CVE-2024-21138
  • Address CVE-2024-27980
  • Address CVE-2024-27982
2024-07-19 12:59 session
  • Address CVE-2024-27983
  • Address CVE-2024-21147
  • Address CVE-2024-21145
  • Address CVE-2024-21140
  • Address CVE-2024-21144
  • Address CVE-2024-21131
  • Address CVE-2024-21138
  • Address CVE-2024-27980
  • Address CVE-2024-27982
2024-07-19 12:59 store
  • Address CVE-2024-27983
  • Address CVE-2024-21147
  • Address CVE-2024-21145
  • Address CVE-2024-21140
  • Address CVE-2024-21144
  • Address CVE-2024-21131
  • Address CVE-2024-21138
  • Address CVE-2024-27980
  • Address CVE-2024-27982
2024-07-17 15:53 svc
  • Fix issue where a sufficiently high enough number of concurrent /publish calls could cause 500s with an exception containing "Unable to load credentials from any of the providers in the chain AwsCredentialsProviderChain(credentialsProviders"
2024-07-12 13:10 ops
  • No user facing changes
2024-07-12 13:10 svc
  • No user facing changes
2024-07-12 13:10 session
  • No user facing changes
2024-07-12 13:10 store
  • No user facing changes
2024-07-05 11:05 ops
  • No user facing changes
2024-07-05 11:05 svc
  • No user facing changes
2024-07-05 11:05 session
  • No user facing changes
2024-07-05 11:05 store
  • No user facing changes
2024-06-28 09:43 ops
  • No user facing changes
2024-06-28 09:43 svc
  • No user facing changes
2024-06-28 09:43 session
  • No user facing changes
2024-06-28 09:43 store
  • No user facing changes
2024-06-21 14:39 ops
  • No user facing changes
2024-06-21 14:39 svc
  • No user facing changes
2024-06-21 14:39 session
  • No user facing changes
2024-06-21 14:39 store
  • No user facing changes
2024-06-14 09:41 ops
  • No user facing changes
2024-06-14 09:41 svc
  • No user facing changes
2024-06-14 09:41 session
  • No user facing changes
2024-06-14 09:41 store
  • No user facing changes
2024-06-07 16:03 ops
  • No user facing changes
2024-06-07 16:03 svc
  • No user facing changes
2024-06-07 16:03 session
  • No user facing changes
2024-06-07 16:03 store
  • No user facing changes
2024-05-31 20:39 ops
  • No user facing changes
2024-05-31 20:39 svc
  • No user facing changes
2024-05-31 20:39 session
  • No user facing changes
2024-05-31 20:39 store
  • No user facing changes
2024-05-24 15:53 ops
  • No user facing changes
2024-05-24 15:53 svc
  • No user facing changes
2024-05-24 15:53 session
  • No user facing changes
2024-05-24 15:53 store
  • No user facing changes
2024-05-17 14:45 ops
  • No user facing changes
2024-05-17 14:45 svc
  • No user facing changes
2024-05-17 14:45 session
  • No user facing changes
2024-05-17 14:45 store
  • No user facing changes
2024-05-10 09:53 ops
  • Address CVE-2024-4317
2024-05-10 09:53 svc
  • Address CVE-2024-4317
2024-05-10 09:53 session
  • No user facing changes
2024-05-10 09:53 store
  • No user facing changes
2024-05-03 10:43 ops
  • No user facing changes
2024-05-03 10:43 svc
  • No user facing changes
2024-05-03 10:43 session
  • No user facing changes
2024-05-03 10:43 store
  • No user facing changes
2024-04-26 13:19 ops
  • No user facing changes
2024-04-26 13:19 svc
  • No user facing changes
2024-04-26 13:19 session
  • No user facing changes
2024-04-26 13:19 store
  • No user facing changes
2024-04-26 10:18 reports
  • No user facing changes
2024-04-19 10:14 ops
  • Address CVE-2023-22043
  • Address CVE-2023-22041
  • Address CVE-2023-22051
  • Address CVE-2023-25193
  • Address CVE-2023-22044
  • Address CVE-2023-22045
  • Address CVE-2023-22049
  • Address CVE-2023-22036
  • Address CVE-2023-22006
2024-04-19 10:14 svc
  • Address CVE-2023-22043
  • Address CVE-2023-22041
  • Address CVE-2023-22051
  • Address CVE-2023-25193
  • Address CVE-2023-22044
  • Address CVE-2023-22045
  • Address CVE-2023-22049
  • Address CVE-2023-22036
  • Address CVE-2023-22006
2024-04-19 10:14 session
  • Address CVE-2023-22043
  • Address CVE-2023-22041
  • Address CVE-2023-22051
  • Address CVE-2023-25193
  • Address CVE-2023-22044
  • Address CVE-2023-22045
  • Address CVE-2023-22049
  • Address CVE-2023-22036
  • Address CVE-2023-22006
2024-04-19 10:14 store
  • No user facing changes
2024-04-12 10:26 ops
  • No user facing changes
2024-04-12 10:26 svc
  • No user facing changes
2024-04-12 10:26 session
  • No user facing changes
2024-04-12 10:26 store
  • No user facing changes
2024-04-05 16:57 ops
  • No user facing changes
2024-04-05 16:57 svc
  • No user facing changes
2024-04-05 16:57 session
  • No user facing changes
2024-04-05 16:57 store
  • No user facing changes
2024-03-29 10:39 ops
  • No user facing changes
2024-03-29 10:39 svc
  • No user facing changes
2024-03-29 10:39 session
  • No user facing changes
2024-03-29 10:39 store
  • No user facing changes
2024-03-22 10:55 ops
  • No user facing changes
2024-03-22 10:55 svc
  • No user facing changes
2024-03-22 10:55 session
  • No user facing changes
2024-03-22 10:55 store
  • No user facing changes
2024-03-15 09:42 ops
  • No user facing changes
2024-03-15 09:42 svc
  • No user facing changes
2024-03-15 09:42 session
  • No user facing changes
2024-03-15 09:42 store
  • No user facing changes
2024-03-08 09:23 ops
  • No user facing changes
2024-03-08 09:23 svc
  • No user facing changes
2024-03-08 09:23 session
  • No user facing changes
2024-03-08 09:23 store
  • No user facing changes
2024-03-01 10:35 ops
  • Address CVE-2024-1597
2024-03-01 10:35 svc
  • Address CVE-2024-1597
2024-03-01 10:35 session
  • No user facing changes
2024-03-01 10:35 store
  • No user facing changes
2024-02-23 15:25 ops
  • Address CVE-2024-0985
2024-02-23 15:25 svc
  • Address CVE-2024-0985
2024-02-23 15:25 session
  • No user facing changes
2024-02-23 15:25 store
  • No user facing changes
2023-02-16 10:30 ops
  • No user facing changes
2023-02-16 10:30 svc
  • No user facing changes
2023-02-16 10:30 session
  • No user facing changes
2023-02-16 10:30 store
  • No user facing changes
2023-02-09 9:40 ops
  • No user facing changes
2023-02-09 9:40 svc
  • No user facing changes
2023-02-09 9:40 session
  • No user facing changes
2023-02-09 9:40 store
  • No user facing changes
2023-02-02 14:30 ops
  • No user facing changes
2023-02-02 14:30 svc
  • No user facing changes
2023-02-02 14:30 session
  • No user facing changes
2023-02-02 14:30 store
  • No user facing changes
2023-01-26 16:35 ops
  • No user facing changes
2023-01-26 16:35 svc
  • No user facing changes
2023-01-26 16:35 session
  • No user facing changes
2023-01-26 16:35 store
  • No user facing changes
2024-01-19 15:00 ops
  • Address CVE-2023-5072
  • Address CVE-2024-20932
  • Address CVE-2024-20918
  • Address CVE-2024-20952
  • Address CVE-2024-20919
  • Address CVE-2024-20921
  • Address CVE-2024-20926
  • Address CVE-2024-20945
  • Address CVE-2024-20955
2024-01-19 15:00 svc
  • Address CVE-2023-5072
  • Address CVE-2024-20932
  • Address CVE-2024-20918
  • Address CVE-2024-20952
  • Address CVE-2024-20919
  • Address CVE-2024-20921
  • Address CVE-2024-20926
  • Address CVE-2024-20945
  • Address CVE-2024-20955
2024-01-19 15:00 session
  • Address CVE-2023-5072
  • Address CVE-2024-20932
  • Address CVE-2024-20918
  • Address CVE-2024-20952
  • Address CVE-2024-20919
  • Address CVE-2024-20921
  • Address CVE-2024-20926
  • Address CVE-2024-20945
  • Address CVE-2024-20955
2024-01-19 15:00 store
  • Address CVE-2023-5072
  • Address CVE-2024-20932
  • Address CVE-2024-20918
  • Address CVE-2024-20952
  • Address CVE-2024-20919
  • Address CVE-2024-20921
  • Address CVE-2024-20926
  • Address CVE-2024-20945
  • Address CVE-2024-20955
2024-01-12 16:10 ops
  • No user facing changes
2024-01-12 16:10 svc
  • No user facing changes
2024-01-12 16:10 session
  • No user facing changes
2024-01-12 16:10 store
  • No user facing changes
2024-01-05 10:20 ops
  • No user facing changes
2024-01-05 10:20 svc
  • No user facing changes
2024-01-05 10:20 session
  • No user facing changes
2024-01-05 10:20 store
  • No user facing changes
2023-12-29 12:55 ops
  • No user facing changes
2023-12-29 12:55 svc
  • No user facing changes
2023-12-29 12:55 session
  • No user facing changes
2023-12-29 12:55 store
  • No user facing changes
2023-12-22 17:30 ops
  • No user facing changes
2023-12-22 17:30 svc
  • No user facing changes
2023-12-22 17:30 session
  • No user facing changes
2023-12-22 17:30 store
  • No user facing changes
2023-12-15 10:10 ops
  • No user facing changes
2023-12-15 10:10 svc
  • No user facing changes
2023-12-15 10:10 session
  • No user facing changes
2023-12-15 10:10 store
  • No user facing changes
2023-12-08 10:10 ops
  • No user facing changes
2023-12-08 10:10 svc
  • No user facing changes
2023-12-08 10:10 session
  • No user facing changes
2023-12-08 10:10 store
  • No user facing changes
2023-12-01 18:45 ops
  • Address CVE-2023-45648
2023-12-01 18:45 svc
  • Address CVE-2023-45648
2023-12-01 18:45 session
  • Address CVE-2023-45648
2023-12-01 18:45 store
  • No user facing changes
2023-11-25 14:50 ops
  • No user facing changes
2023-11-25 14:50 svc
  • No user facing changes
2023-11-25 14:50 session
  • No user facing changes
2023-11-25 14:50 store
  • No user facing changes
2023-11-17 14:30 ops
  • Address CVE-2023-5868
  • Address CVE-2023-5869
  • Address CVE-2023-5870
2023-11-17 14:30 svc
  • Address CVE-2023-5868
  • Address CVE-2023-5869
  • Address CVE-2023-5870
2023-11-17 14:30 session
  • No user facing changes
2023-11-17 14:30 store
  • No user facing changes
2023-11-10 09:15 ops
  • No user facing changes
2023-11-10 09:15 svc
  • No user facing changes
2023-11-10 09:15 session
  • No user facing changes
2023-11-10 09:15 store
  • No user facing changes
2023-11-03 12:45 ops
  • No user facing changes
2023-11-03 12:45 svc
  • No user facing changes
2023-11-03 12:45 session
  • No user facing changes
2023-11-03 12:45 store
  • No user facing changes
2023-10-27 14:45 ops
  • No user facing changes
2023-10-27 14:45 svc
  • No user facing changes
2023-10-27 14:45 session
  • No user facing changes
2023-10-27 14:45 store
  • No user facing changes
2023-10-20 23:00 ops
  • No user facing changes
2023-10-20 23:00 svc
  • No user facing changes
2023-10-20 23:00 session
  • No user facing changes
2023-10-20 23:00 store
  • No user facing changes
2023-10-14 10:10 ops
  • No user facing changes
2023-10-14 10:10 svc
  • No user facing changes
2023-10-14 10:10 session
  • No user facing changes
2023-10-14 10:10 store
  • No user facing changes
2023-10-09 21:00 ops
  • No user facing changes
2023-10-09 21:00 svc
  • No user facing changes
2023-10-09 21:00 session
  • No user facing changes
2023-10-09 21:00 store
  • No user facing changes
2023-09-29 15:18 ops
  • No user facing changes
2023-09-29 15:18 svc
  • Behavior change: remove _version_ attribute from all /query output (this field was an internal implementation detail and had no practical use to end users)
2023-09-29 15:18 session
  • No user facing changes
2023-09-29 15:18 store
  • No user facing changes
2023-09-26 15:40 ops
  • No user facing changes
2023-09-22 22:28 svc
  • Fix bug where /modify was not properly doing typical schema related transformations that /publish performs
  • Fix bug where /get could return the wrong doc in some cases (this was a regression from the release prior in the day)
2023-09-22 12:30 ops
  • Address CVE-2023-41080
2023-09-22 12:30 svc
  • Address CVE-2023-41080
  • Add new /modify endpoint for direct lake modification Rights: Requires publish right for inserts/updates and the delete right for deletes Parameters: "e" a list of entities to be persisted ot the datalake; entities with an _m1key will be updated entities without an _m1key will be inserted "deleted" a list of _m1key values whose entities will be deleted Returns: A map containing the # of documents updated, deleted, and the lowest _m1key of inserted docs
2023-09-22 12:30 session
  • Address CVE-2023-41080
2023-09-22 12:30 store
  • No user facing changes
2023-09-15 16:30 ops
  • No user facing changes
2023-09-15 16:30 svc
  • No user facing changes
2023-09-15 16:30 session
  • No user facing changes
2023-09-15 16:30 store
  • No user facing changes
2023-09-08 12:40 ops
  • No user facing changes
2023-09-08 12:40 svc
  • No user facing changes
2023-09-08 12:40 session
  • No user facing changes
2023-09-08 12:40 store
  • No user facing changes
2023-09-01 11:10 ops
  • No user facing changes
2023-09-01 11:10 svc
  • No user facing changes
2023-09-01 11:10 session
  • No user facing changes
2023-09-01 11:10 store
  • No user facing changes
2023-08-25 16:05 ops
  • Address CVE-2023-39417
  • Address CVE-2023-39418
2023-08-25 16:05 svc
  • Address CVE-2023-39417
  • Address CVE-2023-39418
  • Add cpu utilization in health check
2023-08-25 16:05 session
  • Add cpu utilization in health check
2023-08-25 16:05 store
  • Add cpu utilization in health check
2023-08-18 12:25 ops
  • No user facing changes
2023-08-18 12:25 svc
  • No user facing changes
2023-08-18 12:25 session
  • No user facing changes
2023-08-18 12:25 store
  • No user facing changes
2023-08-11 16:05 ops
  • No user facing changes
2023-08-11 16:05 svc
  • No user facing changes
2023-08-11 16:05 session
  • No user facing changes
2023-08-11 16:05 store
  • No user facing changes
2023-08-06 15:40 ops
  • No user facing changes
2023-08-06 15:40 svc
  • No user facing changes
2023-08-06 15:40 session
  • No user facing changes
2023-08-06 15:40 store
  • No user facing changes
2023-07-30 13:40 ops
  • No user facing changes
2023-07-30 13:40 svc
  • No user facing changes
2023-07-30 13:40 session
  • No user facing changes
2023-07-30 13:40 store
  • No user facing changes
2023-07-28 21:00 svc
  • Enhancement to /query jsonl behavior -- the first line returned will not only include the information from the responseHeader node available in the json response but will also now additionally include all metadata fields from the response node
2023-07-28 17:45 svc
  • Change in behavior to /query when using the jsonl format -- the first line will now be the responseHeader normally returned with the json format.
2023-07-24 19:20 ops
  • Address CVE-2023-22043
  • Address CVE-2023-22041
  • Address CVE-2023-22051
  • Address CVE-2023-25193
  • Address CVE-2023-22044
  • Address CVE-2023-22045
  • Address CVE-2023-22049
  • Address CVE-2023-22036
  • Address CVE-2023-22006
2023-07-24 19:20 svc
  • Truncate query in error messages that echo query; makes error messages involving large queries much more readable
  • Address CVE-2023-22043
  • Address CVE-2023-22041
  • Address CVE-2023-22051
  • Address CVE-2023-25193
  • Address CVE-2023-22044
  • Address CVE-2023-22045
  • Address CVE-2023-22049
  • Address CVE-2023-22036
  • Address CVE-2023-22006
2023-07-24 19:20 session
  • Address CVE-2023-22043
  • Address CVE-2023-22041
  • Address CVE-2023-22051
  • Address CVE-2023-25193
  • Address CVE-2023-22044
  • Address CVE-2023-22045
  • Address CVE-2023-22049
  • Address CVE-2023-22036
  • Address CVE-2023-22006
2023-07-24 19:20 store
  • Address CVE-2023-22043
  • Address CVE-2023-22041
  • Address CVE-2023-22051
  • Address CVE-2023-25193
  • Address CVE-2023-22044
  • Address CVE-2023-22045
  • Address CVE-2023-22049
  • Address CVE-2023-22036
  • Address CVE-2023-22006
2023-07-18 19:15 ops
  • Fix bug where large POST payloads (>2MB) would fail
2023-07-18 19:15 svc
  • Fix bug where large POST payloads (>2MB) would fail
2023-07-18 19:15 session
  • Fix bug where large POST payloads (>2MB) would fail
2023-07-18 19:15 store
  • No user facing changes
2023-07-16 16:15 ops
  • Address CVE-2023-34981
2023-07-16 16:15 svc
  • Address CVE-2023-34981
2023-07-16 16:15 session
  • Address CVE-2023-34981
2023-07-16 16:15 store
  • No user facing changes
2023-07-13 15:00 svc
  • Fix regression in 2023-07-12 release that would cause svc servers to be unable to connect to store servers.
2023-07-12 19:34 svc
  • /query format parameter now supports jsonl in addition to csv (and the default json); the jsonl and csv types can only used for either documents or facets (not both in one query). In addition only a single facet can be returned in a format other than than json
2023-07-09 17:00 ops
  • Fix bug where admin param was required for account/list; this regression occured in the 2023-06-05 release
  • Support POST in addition to GET for account/list, env/list
  • Make rights parameter in account/user/add not required
  • Address CVE-2021-23214
  • Address CVE-2021-23222
  • Address CVE-2021-32027
  • Address CVE-2021-32028
  • Address CVE-2021-32029
  • Address CVE-2022-1552
  • Address CVE-2022-2625
  • Address CVE-2022-41862
  • Address CVE-2023-2454
  • Address CVE-2023-2455
2023-07-09 17:00 svc
  • No user facing changes
2023-07-09 17:00 session
  • No user facing changes
2023-07-09 17:00 store
  • No user facing changes
2023-07-04 09:10 ops
  • Change a couple of http error codes from 409 (CONFLICT) to 400 (BAD REQUEST)
2023-07-04 09:10 svc
  • No user facing changes
2023-07-04 09:10 session
  • No user facing changes
2023-07-04 09:10 store
  • No user facing changes
2023-06-27 20:50 ops
  • No user facing changes
2023-06-27 20:50 svc
  • No user facing changes
2023-06-27 20:50 session
  • No user facing changes
2023-06-27 20:50 store
  • No user facing changes
2023-06-26 13:26 reports
  • Fix bug where incorrect # of documents would get exported when the number of documents to export exceeded 1,000,000 documents
2023-06-21 10:41 ops
  • Fix bug with env/rescale ZDD deployments; previously a rescale may have caused a temporary outage observed as 503 errors
  • Make various environment related methods return a 400 in the case of a user error rather than a 409
2023-06-21 10:41 svc
  • Fix a regression introduced in 2023-06-05 release that broke CORS
  • Better error message and status code in store/add and session/add when trying to add a store that already exists
2023-06-21 10:41 session
  • No user facing changes
2023-06-21 10:41 store
  • No user facing changes
2023-06-20 17:06 reports
  • Fix bug where incorrect number of documents were getting exported
2023-06-05 10:15 ops
  • General performance improvements
2023-06-05 10:15 svc
  • General performance improvements
  • /next, /store/list, /session/list, /token/list, /health endpoints can now accept POST requests in addition to GETs
2023-06-05 10:15 session
  • General performance improvements
2023-06-05 10:15 store
  • No user facing changes
2023-05-11 22:15 ops
  • No user facing changes
2023-05-11 22:15 svc
  • Better error messages for data store related issues
  • Better error messages when an extremely long string can't be coerced into a an expected type; previously the long string would be truncated and any characters after the first 80 would be omitted. Now the first 40 and last 40 characters are shown.
  • /health now shows memory usage for svc, data stores, and session stores
  • /health now shows disk usage for data stores
  • Rename weightedAvg aggregator to avg; weightedAvg is deprecated and will be removed in a future release
  • The previous behavior of avg, which returned the avg of unique values of a field, can be obtained by a new aggregator uniqueAvg
2023-05-11 22:15 store
  • Provide disk/memory information exposed via environment /health service
2023-05-11 22:15 session
  • Provide memory information exposed via environment /health service
2023-05-01 20:30 ops
  • No user facing changes
2023-05-01 20:30 svc
  • No user facing changes
2023-05-01 20:30 session
  • No user facing changes
2023-05-01 09:50 svc
  • Change in behavior regarding boolean types:
  • Previously all values other than the string "true" (including null) were interpreted as false;
  • Now null is simply null and consistent with other data types
  • In addition to null, valid boolean values are "true", "1", "t", "y", "yes", "false", "0", "f", "n", "no", 1, and 0 and any cased variations
  • Publishing a boolean value not in that valid set will cause an error if publish-permissive is false or do nothing if publish-permissive is true (this is consistent with behavior of other data types)
2023-04-23 13:30 svc
  • New system property property-source: when populated, published documents will have an additional attribute indicating the source (typically an s3 file) of the document.
  • Add system property property-unknown: when publish-permissive is true and a document was only partially loaded, property-unknown names a property that will contain all properties that were not loaded because the property was not present in the schema.
  • Add system property property-error: when publish-permissive is true and a document was only partially loaded, property-error names a property that will contain all properties that were not loaded for reasons other than a missing schema element; this will generally be because of data type mismatches ("xyz" for a date or numeric field, 1.2 for a double field, etc.)
2023-04-18 22:30 ops
  • No user facing changes
2023-04-18 22:30 svc
  • No user facing changes
2023-04-18 22:30 session
  • No user facing changes
2023-04-16 19:00 svc
  • Add store/rename and session/rename endpoints
  • Raise error if store param is missing for /session/key
  • Fix a couple of potential resource leaks related to db resources
  • Better handling of OOM situations; a service that runs into an OOM situation is killed so that it can be brought back up cleanly
2023-04-16 19:00 session
  • Fix a couple of potential resource leaks related to db resources
  • Better handling of OOM situations; a service that runs into an OOM situation is killed so that it can be brought back up cleanly
2023-04-16 19:00 ops
  • Add env/store/rename and /env/session/rename endpoints
  • Environment TLS policy now disallows tls 1 and tls1.1; services now rated an A by ssllabs.com
  • Fix a couple of potential resource leaks related to db resources
  • Better handling of OOM situations; a service that runs into an OOM situation is killed so that it can be brought back up cleanly
2023-03-29 10:05 svc
  • Bulletproof geo appending when matched geo data does not contain latitude/longitude data
2023-03-28 20:55 svc
  • Bulletproof geo appending when client supplies ip address that is not a true ip address
2023-03-25 21:30 ops
  • More helpful error messages when request params fail validation
  • Store names can only have lower case letter, numbers, or hyphens; uppercase letters are no longer allowed
2023-03-25 21:30 svc
  • More helpful error messages when request params fail validation
  • Store names can only have lower case letter, numbers, or hyphens; uppercase letters are no longer allowed
  • Fix bug that was thought to be fixed in 2023-03-10 release involving sending back the query params in the response to /query.
2023-03-25 21:30 session
  • No specific session store fixes but rebuilding image because of a number of changes with common code
2023-03-16 19:30 reports
  • Small layout fix
2023-03-13 18:00 reports
  • Fix some bugs in 2023-03-10 deployment
  • Change to support svc breaking change from 2023-03-10
2023-03-10 18:00 reports
  • Support brandability: Override brand.css and brand.js in the theme directory.
2023-03-10 03:42 svc
  • Fix bug where /query was not sending back exact params sent to it
  • Breaking change: /query response responseHeader.params.json was previously the raw json passed in; this string is now json parsed
2023-03-09 18:00 reports
  • Allow Ctrl+F style searchability of column headers
  • Fixed bug returning rows in case of facets query.
2023-03-09 14:30 svc
  • Fix /query NPE bug if facet keys contain periods
2023-03-08 13:20 svc
  • Fix bug where session store key was not respected
  • Environment data and session store definitions are now cached and refreshed every 5 seconds; note when using /store/* and /session/* services. This is to improve performance in for high volume usages of /insert or /session/update
  • Support yyyy-mm-dd style dates
  • Date related formatters in transform rules sets now use Java 8 DateTimeFormatter syntax instead of SimpleDateFormat syntax; DateTimeFormatter provides a superset of functionality and is backwards compatible
  • Substantial json serialization performance improvement
  • Support countUnique/avg/weightedAvg facet aggregators in /query
  • Better error messages around /query; ignore (rather than error) unhandled params
2023-03-08 13:20 session
  • No user facing changes
2023-03-07 14:00 store
  • Fix bug introduced in 2023-02-27 store release that would prevent a boost store from getting created (related to dynamically setting memory usage)
2023-02-27 21:00 store
  • Default default search operator back to OR (from AND) (this changed somewhat accidentally in Dec-2023)
2023-02-26 18:00 ops
  • Substantial json serialization performance improvement
2023-02-10 10:25 ops
  • Error message grammar fix when unable to create an environment