Improve AI - Easily Optimize Your App with Reinforcement Learning

Optimize and personalize your apps with fast AI decisions that get smarter over time. Decisions are made immediately, on-device, with no network latency.

The heart of Improve AI is the which() statement. which() is like an AI if/then statement.

greeting = greetingsModel.which("Hello", "Howdy", "Hola")

which() uses logic stored in a machine learning model to make the best choice. Models are easily trained with reinforcement learning.

if (success) {
    greetingsModel.addReward(1.0)
}

With reinforcement learning, positive rewards are assigned for positive outcomes (a “carrot”) and negative rewards are assigned for undesirable outcomes (a “stick”).

which() chooses the option that provides the highest expected reward given the current conditions.

When rewards are business metrics, such as revenue, conversions, or user retention, the decisions will optimize to automatically improve those metrics over time.

That’s like A/B testing on steroids.

However, unlike A/B testing, Improve AI uses context to personalize each decision. On iOS and Android, the context automatically includes language, OS, device type, day of week, time of day, app version, and many other attributes. (No personally identifiable information is ever used, transmitted, or stored)

With this context, for a Spanish speaker the greetings model will learn to choose Hola.

Custom context can also be provided via given():

greeting = greetingsModel.given({"language": "cowboy"})
                         .which("Hello", "Howdy", "Hola")

Given the language is cowboy, the variant with the highest expected reward is Howdy and the model will learn to make that choice.

The combination of given(), which(), and addReward() make it simple to implement personalization, recommendations, dynamic In App Offers, and more.

Example: Personalizing an Upsell Offer

Improve AI is powerful and flexible. which() accepts any JSON encodeable object including strings, numbers, booleans, lists, and dictionaries.

For a dungeon crawler game, say the user is purchasing an item using an In App Purchase. Improve AI can choose an additional product to display as an upsell offer during checkout. With a few lines of code, a model can be trained that will learn to optimize the upsell offer given the original product being purchased.

product = { "name": "red sword", "price": 4.99 }

upsellOffers = [{ "name": "gold", "quantity": 100, "price": 1.99 },
                { "name": "diamonds", "quantity": 10, "price": 2.99 },
                { "name": "red scabbard", "price": 0.99 }]

upsell = upsellModel.given(product)
                    .which(upsellOffers)

The product to be purchased is the red sword. Notice that the variants are dictionaries with a mix of string and numeric values.

The rewards in this case might be any additional revenue from the upsell.

if (upsellPurchased) {
    upsellModel.addReward(upsell.price)
}

While it is reasonable to hypothesize that the red scabbord might be the best upsell offer to pair with the red sword, it is still a guess. Any time a guess is made on the value of a variable, instead use Improve AI to decide.

Replace guesses with AI decisions.

Example: Performance Tuning

I used to write a lot of video streaming code. The initial motivation for Improve AI came out of my frustrations with attempting to tune video streaming clients across heterogenious networks.

I was forced to make guesses on performance sensitive configuration defaults through slow trial and error. My client configuration code maybe looked something like this:

config = { "bufferSize": 2048,
           "videoBitrate": 384000 }

This is the code I wish I could have written:

config = configModel.which({"bufferSize": [1024, 2048, 4096, 8192],
                            "videoBitrate": [256000, 384000, 512000]})

This example decides multiple variables simultaneously. Notice that instead of a single list of variants, a dictionary mapping keys to lists of variants is provided to which. This multi-variate mode jointly optimizes all variables for the highest expected reward.

The rewards in this case might be negative to penalize any stalls during video playback.

if (videoStalled) {
    configModel.addReward(-0.001)
}

Improve AI frees us from having to overthink our configuration values during development. We simply give it some reasonable variants and let it learn from real world usage.

This is an enormously powerful tool and I’m sure you’ll come up with some very interesting uses. Look for places where you’re relying on guesses or an executive decision and consider instead directly optimizing for the outcomes you desire.

Improve AI is available now for iOS, Android, and Python. See the Quick-Start Guide to learn more.

Thank you for your efforts to improve the world a little bit today.

– Justin Chapweske

Recent Posts