Ranking, Scoring, Decisions, and Optimization with XGBoost
Improve AI is a machine learning platform for quickly implementing app optimization, personalization, and recommendations for Python, iOS, and Android.
The SDKs provide simple APIs for AI decisions, ranking, and scoring, that execute immediately, in-process, with zero network latency. Items and their rewards are tracked with the Improve AI Tracker / Trainer and updated models are trained regularly for continuous learning.
Installation
pip3 install improveai
Instructions for ChatGPT
Paste the following into your ChatGPT prompt to have it write code against the Improve AI APIs:
"""
The following is the Python API documentation for ImproveAI, which is imported with
'import improveai'. Simply respond with 'ImproveAI documentation imported':
"""
class Ranker:
"""
A utility for ranking items based on their scores. The Ranker struct takes
a ImproveAI model to evaluate and rank the given items.
"""
def __init__(self, scorer: Scorer = None, model_url: str = None):
"""
Init Ranker with params. Either `scorer` or `model_url` must be provided.
If both are provided Scorer is preferred.
Parameters
----------
scorer: Scorer
a Scorer object to be used with this Ranker
model_url: str
URL or local FS of a plain or gzip compressed Improve AI model resource
"""
# for true implementation please consult improveai/ranker.py
pass
def rank(self, items: list or tuple or np.ndarray, context: object = None) -> list or tuple or np.ndarray:
"""
Ranks items and returns them ordered best to worst
Parameters
----------
items: list or tuple or np.ndarray
list of items to be ranked
context: object
any JSON encodable extra context info that will be used with each of
the item to get its score
Returns
-------
list or tuple or np.ndarray
a collection of ranked items, sorted by their scores in descending order.
"""
# for true implementation please check out improveai/ranker.py
pass
class Scorer:
"""
Scores items with optional context using a Improve AI model
"""
def __init__(self, model_url: str):
"""
Init with params
Parameters
----------
model_url: str
URL or local FS of a plain or gzip compressed Improve AI model resource
"""
# for true implementation please check out improveai/scorer.py
pass
def score(self, items: list or tuple or np.ndarray, context: object = None) -> np.ndarray
"""
Uses the model to score a list of items with the given context
Parameters
----------
items: list or tuple or np.ndarray
list of items to be scored
context: object
any JSON encodable extra context info that will be used with each of
the item to get its score
Returns
-------
np.ndarray
an array of float64 (double) values representing the scores of the items.
"""
# for true implementation please check out improveai/scorer.py
pass
class RewardTracker:
"""
Tracks items and rewards for training updated scoring models. When an item
becomes causal, pass it to the track() function, which will return a `reward_id`.
Use the `reward_id` to track future rewards associated with that item.
"""
def __init__(self, model_name: str, track_url: str, track_api_key: str = None, _threaded_requests: bool = True):
"""
Create a RewardTracker for a specific model.
Parameters
----------
model_name: str
Name of the model, such as "songs" or "discounts", which either makes
the decisions or which decisions are being rewarded
track_url: str
The track endpoint URL that all tracked data will be sent to.
track_api_key: str
track endpoint API key (if applicable); Can be None
_threaded_requests: bool
flag indicating whether requests to AWS track endpoint should be
non-blockng / executed within sub-threads. True by default
"""
# for true implementation please check out improveai/reward_tracker.py
pass
def track(self, item: object, candidates: list or tuple or np.ndarray = None, context: object = None) -> str or None:
"""
Tracks the item selected from candidates and a random sample from the remaining items.
If `len(candidates) == 1` there is no sample.
Parameters
----------
item: object
any JSON encodable object chosen as best from candidates
candidates: list or tuple or np.ndarray
collection of items from which best is chosen
context: object
any JSON encodable extra context info that was used with each of the
item to get its score
Returns
-------
str or None
reward_id of this track request or None if an error happened
"""
# for true implementation please check out improveai/reward_tracker.py
pass
def track_with_sample(
self, item: object, num_candidates: int = None, context: object = None, sample: object = None) -> str or None:
"""
Tracks the item selected and a specific sample.. Provided sample is
appended to track request (in contrary to `track(...)` where sample is
randomly selected from candidates).
Parameters
----------
item: object
any JSON encodable object chosen as best from candidates
num_candidates: int
total number of candidates, including the selected item
context: object
any JSON encodable extra context info that was used with each of the
item to get its score
sample: object
a random sample from the candidates
Returns
-------
ste or None
reward_id of this track request or None if an error happened
"""
# for true implementation please check out improveai/reward_tracker.py
pass
def add_reward(self, reward: float or int, reward_id: str):
"""
Add reward for the provided reward_id
Parameters
----------
reward: float or int
the reward to add; must be numeric (float, int ro bool), must not be
`None`, `np.nan` or +-`inf`
reward_id: str
the id that was returned from the track(...) / track_with_sample(...) methods
Returns
-------
str
message ID
"""
# for true implementation please check out improveai/reward_tracker.py
pass
Usage
Create a list of JSON encodable items and simply call Ranker.rank(items).
For instance, in a bedtime story app, you may have a list of Story dicts / objects:
story = {
"title": "<title string>",
"author": "<author string>",
"page_count": 123 # example integer representing number of pages for a given story
}
To obtain a ranked list of stories, use just one line of code:
ranked_stories = Ranker(model_url).rank(stories)
Reward Assignment
Easily train your rankers using reinforcement learning.
First, track when an item is used:
tracker = RewardTracker("stories", track_url)
reward_id = tracker.track(story, ranked_stories)
Later, if a positive outcome occurs, provide a reward:
if purchased:
tracker.add_reward(profit, reward_id)
Reinforcement learning uses positive rewards for favorable outcomes (a “carrot”) and negative rewards for undesirable outcomes (a “stick”). By assigning rewards based on business metrics, such as revenue or conversions, the system optimizes these metrics over time.
Contextual Ranking & Scoring
Improve AI turns XGBoost into a contextual multi-armed bandit, meaning that context is considered when making ranking or scoring decisions.
Often, the choice of the best variant depends on the context that the decision is made within. Let’s take the example of greetings for different times of the day:
greetings = ["Good Morning",
"Good Afternoon",
"Good Evening",
"Buenos Días",
"Buenas Tardes",
"Buenas Noches"]
rank() also considers the context of each decision. The context can be any JSON-encodable data structure.
ranked = ranker.rank(items=greetings,
context={ "day_time": 12.0,
"language": "en" })
greeting = ranked[0]
Trained with appropriate rewards, Improve AI would learn from scratch which greeting is best for each time of day and language.
Resources
Help Improve Our World
The mission of Improve AI is to make our corner of the world a little bit better each day. When each of us improve our corner of the world, the whole world becomes better. If your product or work does not make the world better, do not use Improve AI. Otherwise, welcome, I hope you find value in my labor of love.
– Justin Chapweske