Improve AI Basics - Decisions

painting of a dog trying to decide which button to push
"painting of a boy choosing a flavor of ice cream" by DALL-E

which(…variants)

Improve AI provides a simple, intuitive API for making decisions: the which() statement.

greeting = greetingsModel.which("Hello", "Howdy", "Hola")

which() takes a list of variants and returns the best variant - the “best” being the variant that provides the highest expected reward given the current conditions.

After the decision is returned, rewards are assigned via addReward().

if (success) {
    greetingsModel.addReward(1.0)
}

The decision and its rewards are tracked with the Improve AI Gym, our cloud-based reinforcement learning platform.

Variants

Variants can be any primitive type or object that is encodable as JSON.

As we’ve seen above, variants can be strings.

greeting = greetingsModel.which("Hello", "Howdy", "Hola")

Variants can also be numbers:

discount = discountModel.which(0.1, 0.2, 0.3)

Boolean values:

enabled = featureFlagModel.which(true, false)

Null/None/Nil:

item = filterModel.which(item, nil)

Complex Objects:

themes = [{ "font": "Helvetica", "size": 12, "color": "#000000"  },
          { "font": "Comic Sans", "size": 16, "color": "#F0F0F0" }]

theme = themeModel.which(themes)

For complex objects, all available properties may potentially be used to assist in the decision making process. In particular, numeric properties can be especially powerful in determining which variant is best.

Contextual Decisions

Often, the choice of the best variant depends on the context that the decision is made within. Let’s take the example of greetings for different times of the day:

greeting = greetingsModel.which("Good Morning", 
                                "Good Afternoon", 
                                "Good Evening")

which() also considers the context of each decision. On iOS and Android, the context automatically includes many attributes, including language and time of day.

For Python, context must be explicitly provided:

greeting = greetings_model.given({ "day_time": 12.0,
                                   "language": "en" }) \
                          .which("Good Morning",
                                 "Good Afternoon",
                                 "Good Evening")

given() is particularly powerful for implementing personalized decisions. Imagine we’re writing a wine recommender that personalizes the wine recommendation based on the user’s choice of entree.:

wine = sommelierModel.given(entree).which(wines)

In addition to optimizing lists of variants, which() forms the foundation of more complicated forms of optimization. See Multivariate Optimization for more information on this topic.

Recent Posts

2 minute read

Making Decisions with Ranker

1 minute read

In this tutorial, we’ll walk through how to use the Ranker class to make decisions, specifically choosing a discount to offer from a list of discounts. We’ll...