AndroidEnv - The Android Interaction & Learning Platform

English · 简体中文

AndroidEnv is a Python and Kotlin platform for interacting with Android devices (both virtualized emulators and physical hardware). Built for foundation model UI agents, automated testing, and Reinforcement Learning (RL) research, AndroidEnv provides low-latency device telemetry and control over any Android application.

In AndroidEnv v2, device interaction is decoupled from Reinforcement Learning task definitions:

tests PyPI version Downloads

Index

Platform Features

Getting Started

Installation

Install AndroidEnv from PyPI:

$ python3 -m pip install android-env

Or clone the repository from main and build/test with Bazel (ensure ANDROID_HOME points to a local Android SDK installation):

$ git clone https://github.com/google-deepmind/android_env/
$ cd android_env
$ bazel test //...

To build the PyPI wheel locally using Bazel:

$ bazel build //:android_env_wheel

1. Set Up an Android Device

Connect a physical Android device via USB debugging (adb devices) or create an Android Virtual Device (AVD) using Android Studio. See the Emulator guide for step-by-step instructions.

2. Direct Control with AndroidDevice

For UI agents, foundation model evaluations, and automation scripts that do not need RL episode loops:

from android_env.components import android_device_loader
from android_env.components import config_classes
from android_env.proto import device_env_service_pb2

config = config_classes.AndroidDeviceConfig(serial='emulator-5554')
with android_device_loader.load_android_device(config) as device:
  # Subscribe to screenshots and accessibility trees:
  device.update_subscriptions({
      device_env_service_pb2.DEVICE_SIGNAL_SCREENSHOT,
      device_env_service_pb2.DEVICE_SIGNAL_ACCESSIBILITY_TREE,
  })

  # Read the latest device state:
  state = device.get_state()

  # Tap the center of the screen:
  device.send_action(
      device_env_service_pb2.Action(
          action_type=device_env_service_pb2.ACTION_TYPE_TOUCH_DOWN,
          touch_position=device_env_service_pb2.Point(x=0.5, y=0.5),
      )
  )
  device.send_action(
      device_env_service_pb2.Action(
          action_type=device_env_service_pb2.ACTION_TYPE_TOUCH_UP,
      )
  )

3. Launch the Interactive Web Playground

Explore the device screen, inspect live accessibility trees, and run ADB commands from your browser (after pip install "android-env[playground]"):

$ android-env-playground --serial=emulator-5554

Or launch a local AVD from a source checkout with Bazel:

$ bazel run //android_env/playground:playground -- \
    --config=android_env/playground/configs/emulator_config.py

To expose the device as a stdio Model Context Protocol (MCP) server for LLM clients (after pip install "android-env[mcp]"):

$ android-env-mcp --serial=emulator-5554

4. Reinforcement Learning (dm_env)

To define MDP tasks with rewards and episode resets (after pip install "android-env[rl]"), see the Tasks guide, Example tasks, and detailed instructions.

About

This library is developed and maintained by Google DeepMind. You can find the original technical report on arXiv, as well as the introductory blog post.

If you use AndroidEnv in your research, you can cite the paper using the following BibTeX:

@article{ToyamaEtAl2021AndroidEnv,
  title     = "{AndroidEnv}: A Reinforcement Learning Platform for Android",
  author    = {Daniel Toyama and Philippe Hamel and Anita Gergely and
               Gheorghe Comanici and Amelia Glaese and Zafarali Ahmed and Tyler
               Jackson and Shibl Mourad and Doina Precup},
  year      = {2021},
  eprint    = {2105.13231},
  archivePrefix = {arXiv},
  primaryClass = {cs.LG},
  volume    = {abs/2105.13231},
  url       = {http://arxiv.org/abs/2105.13231},
}

Disclaimer: This is not an official Google product.