Improve AI Quick Start

This guide covers setting up the Improve AI Trainer, Gym, and SDKs to track decisions and rewards, and train updated models.

For a deeper treatment of the Improve AI concepts see the documents on Reinforcement Learning, Decisions, Ranking, Scoring and Multivariate Optimization.

Subscribe to a Trainer

  • To train unlimited decision models on up to 100,000 decisions each, subscribe to the free trainer.
  • To train decision models on over 100,000 decisions, subscribe to the pro trainer.

Deploy the Gym

The Improve AI Gym is a scalable stack of serverless components that runs on AWS under your full control.

The Gym provides three primary functions:

  • An HTTPS endpoint for tracking decisions and rewards from client SDKs.
  • A store of logged decisions and rewards on AWS S3.
  • Scheduling and configuration for one or more trainers to train updated decision models.

For now we suggest installing the gym by making a private fork of the repo. We are also considering packaging the gym as an NPM package or a Serverless Framework Plugin. Please send suggestions for deployment packaging to [email protected]

Install the Serverless Framework

$ npm install -g serverless

Install NPM Dependencies

$ npm install

Configure Gym and Model Training

$ nano config/config.yml

Edit the organization and project name. AWS resource names depend on these values and must be globally unique.

organization: acme
project: demo

Paste your trainer image url from your trainer subscription.

images:
  free: # ⬅️ !!! Subscribe at https://aws.amazon.com/marketplace/pp/prodview-pyqrpf5j6xv6g and paste image url
  pro: # ⬅️ !!! Subscribe at https://aws.amazon.com/marketplace/pp/prodview-adchtrf2zyvow and paste image url

Configure the decision models to train

models:
  offers:
  config:
  feed:
  search-rerank:

Deploy the Gym

Deploy the gym to a new dev stage in us-east-1

$ serverless deploy --stage dev

Deploying improveai-acme-demo to stage dev (us-east-1)

✔ Service deployed to stack improveai-acme-demo-dev (111s)

endpoint: https://xxxx.lambda-url.us-east-1.on.aws/

The output of the deployment will list the track endpoint URL like https://xxxx.lambda-url.us-east-1.on.aws. The track endpoint URL may be used directly by the client SDKs to track decisions and rewards. Alternately, a CDN or proxy may be configured in front of the track endpoint URL for greater administrative control.

Model Training

The deployment will create a models S3 bucket in the form of improveai-{organization}-{project}-{stage}-models. After each round of training, updated models are automatically uploaded to the models bucket.

The models bucket is private by default. Make the /models/latest/ directory public to serve models directly from S3. Alternatively, a CDN may be configured in front of the models S3 bucket.

Model URLs follow the template of https://{modelsBucket}.s3.amazonaws.com/models/latest/{modelName}.{mlmodel|xgb}.gz. The Android and Python SDKs use .xgb.gz models and the iOS SDK uses .mlmodel.gz models.

Note: Until some decisions have been tracked, model training will fail. This is expected behavior.

Integrate a Client Library

Integrate a client library to make decisions and assign rewards.

See their individual documentation to get started:

Recent Posts