Optimize Any Python, Swift, or Java Object with Reinforcement Learning
Improve AI is a machine learning platform for making apps self-improving, meaning they optimize their own data structures and variables to improve revenue an...
This guide covers setting up the Improve AI Trainer, Tracker, and SDKs to track decisions and rewards, and train updated models.
For a deeper treatment of the Improve AI concepts see the documents on Reinforcement Learning, Decisions, Ranking, Scoring and Multivariate Optimization.
The Improve AI Tracker is a scalable stack of serverless components that runs on AWS under your full control.
The Tracker provides three primary functions:
For now we suggest installing the gym by making a private fork of the repo. We are also considering packaging the gym as an NPM package or a Serverless Framework Plugin. Please send suggestions for deployment packaging to [email protected]
$ npm install -g serverless
$ npm install
$ nano config/config.yml
Edit the organization and project name. AWS resource names depend on these values and must be globally unique.
organization: acme
project: demo
Paste your trainer image url from your trainer subscription.
images:
free: # ⬅️ !!! Subscribe at https://aws.amazon.com/marketplace/pp/prodview-pyqrpf5j6xv6g and paste image url
pro: # ⬅️ !!! Subscribe at https://aws.amazon.com/marketplace/pp/prodview-adchtrf2zyvow and paste image url
Configure the models to train
models:
offers:
config:
feed:
search-rerank:
Deploy the tracker to a new dev stage in us-east-1
$ serverless deploy --stage dev
Deploying improveai-acme-demo to stage dev (us-east-1)
✔ Service deployed to stack improveai-acme-demo-dev (111s)
endpoint: https://xxxx.lambda-url.us-east-1.on.aws/
The output of the deployment will list the track endpoint URL like https://xxxx.lambda-url.us-east-1.on.aws. The track endpoint URL may be used directly by the client SDKs to track decisions and rewards. Alternately, a CDN or proxy may be configured in front of the track endpoint URL for greater administrative control.
The deployment will create a models S3 bucket in the form of improveai-{organization}-{project}-{stage}-models. After each round of training, updated models are automatically uploaded to the models bucket.
The models bucket is private by default. Make the /models/latest/ directory public to serve models directly from S3. Alternatively, a CDN may be configured in front of the models S3 bucket.
Model URLs follow the template of https://{modelsBucket}.s3.amazonaws.com/models/latest/{modelName}.{mlmodel|xgb}.gz. The Android and Python SDKs use .xgb.gz models and the iOS SDK uses .mlmodel.gz models.
Note: Until some decisions have been tracked, model training will fail. This is expected behavior.
Integrate a client library to make decisions and assign rewards.
See their individual documentation to get started:
Improve AI is a machine learning platform for making apps self-improving, meaning they optimize their own data structures and variables to improve revenue an...
I’ve been working on Improve AI for many years, but by 2017 the idea had begun to gel enough that I was ready to talk about it.
Optimize and personalize your apps with fast AI decisions that get smarter over time. Decisions are made immediately, on-device, with no network latency. Th...
In 2012, Steve Hanov wrote the popular and controversial blog post “20 lines of code that will beat A/B testing every time” that brought the previously acade...