UIAutomator2.0和AccessibilityService实现分析

发表于 更新于 分类于 阅读次数：

0x00 前言

UiAutomator是Android 4.1以上提供的一个UI自动化测试工具，4.3升级到了UiAutomator2.0，实现方式也从UiTestAutomationBridge变成了UiAutomation。

0x01 UiAutomation实现分析

UiAutomation类位于android.app包下面，是API18新增的类。

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
public final class UiAutomation {
    private final IAccessibilityServiceClient mClient;
    private final IUiAutomationConnection mUiAutomationConnection;

    public void connect();
    public void disconnect();
    public final boolean performGlobalAction(int action);
    public final AccessibilityServiceInfo getServiceInfo();
    public AccessibilityNodeInfo getRootInActiveWindow();
    public boolean injectInputEvent(InputEvent event, boolean sync);
    public boolean setRotation(int rotation);
    public AccessibilityEvent executeAndWaitForEvent(Runnable command, AccessibilityEventFilter filter, long timeoutMillis);
    public void waitForIdle(long idleTimeoutMillis, long globalTimeoutMillis);
    public Bitmap takeScreenshot();
    public void setRunAsMonkey(boolean enable);
｝

以上是UiAutomation类中最重要的成员变量和函数，所有的接口基本都是通过调用mClient和mUiAutomationConnection这两个对象实现的。

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
public UiAutomation(Looper looper, IUiAutomationConnection connection) {
    if (looper == null) {
        throw new IllegalArgumentException("Looper cannot be null!");
    }
    if (connection == null) {
        throw new IllegalArgumentException("Connection cannot be null!");
    }
    mUiAutomationConnection = connection;
    mClient = new IAccessibilityServiceClientImpl(looper);
}

interface IUiAutomationConnection {
    void connect(IAccessibilityServiceClient client);
    void disconnect();
    boolean injectInputEvent(in InputEvent event, boolean sync);
    boolean setRotation(int rotation);
    Bitmap takeScreenshot(int width, int height);
    boolean clearWindowContentFrameStats(int windowId);
    WindowContentFrameStats getWindowContentFrameStats(int windowId);
    void clearWindowAnimationFrameStats();
    WindowAnimationFrameStats getWindowAnimationFrameStats();
    void executeShellCommand(String command, in ParcelFileDescriptor fd);
    void grantRuntimePermission(String packageName, String permission, int userId);    
    void revokeRuntimePermission(String packageName, String permission, int userId);

    // Called from the system process.
    oneway void shutdown();
}

UiAutomation构造函数需要传入两个参数，一个是线程Looper对象，用于发送消息，一个是IUiAutomationConnection接口实例。

1
2
3
4
5
6
7
8
9
public void connect() {
    if (mHandlerThread.isAlive()) {
        throw new IllegalStateException("Already connected!");
    }
    mHandlerThread.start();
    mUiAutomation = new UiAutomation(mHandlerThread.getLooper(),
            new UiAutomationConnection());
    mUiAutomation.connect();
}

这是UiAutomationShellWrapper类中的一个方法，正好可以看到UiAutomation如何初始化。构造函数的第二个参数实际使用的是UiAutomationConnection类实例，这个类也是在android.app包下面，正是继承自IUiAutomationConnection.Stub类。

UiAutomation中一个重要的成员变量是mClient，它的类型是IAccessibilityServiceClient。

1
2
3
4
5
6
7
8
9
10
11
12
13
14
oneway interface IAccessibilityServiceClient {

    void init(in IAccessibilityServiceConnection connection, int connectionId, IBinder windowToken);

    void onAccessibilityEvent(in AccessibilityEvent event);

    void onInterrupt();

    void onGesture(int gesture);

    void clearAccessibilityCache();

    void onKeyEvent(in KeyEvent event, int sequence);
}

IAccessibilityServiceClient是一个AIDL接口定义，在抽象类AccessibilityService中实现了一个该接口的实现类IAccessibilityServiceClientWrapper。UiAutomation中定义了一个IAccessibilityServiceClientWrapper的子类IAccessibilityServiceClientImpl，主要是重写了onAccessibilityEvent方法，用于获取AccessibilityEvent事件。UiAutomation的构造函数中实例化的正是IAccessibilityServiceClientImpl实例。

UiAutomation的初始化过程主要是在connect方法中。

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
/**
    * Connects this UiAutomation to the accessibility introspection APIs.
    *
    * @hide
    */
public void connect() {
    synchronized (mLock) {
        throwIfConnectedLocked();
        if (mIsConnecting) {
            return;
        }
        mIsConnecting = true;
    }

    try {
        // Calling out without a lock held.
        mUiAutomationConnection.connect(mClient);
    } catch (RemoteException re) {
        throw new RuntimeException("Error while connecting UiAutomation", re);
    }

    synchronized (mLock) {
        final long startTimeMillis = SystemClock.uptimeMillis();
        try {
            while (true) {
                if (isConnectedLocked()) {
                    break;
                }
                final long elapsedTimeMillis = SystemClock.uptimeMillis() - startTimeMillis;
                final long remainingTimeMillis = CONNECT_TIMEOUT_MILLIS - elapsedTimeMillis;
                if (remainingTimeMillis <= 0) {
                    throw new RuntimeException("Error while connecting UiAutomation");
                }
                try {
                    mLock.wait(remainingTimeMillis);
                } catch (InterruptedException ie) {
                    /* ignore */
                }
            }
        } finally {
            mIsConnecting = false;
        }
    }
}

这里主要是调用了mUiAutomationConnection.connect(mClient)。

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
public void connect(IAccessibilityServiceClient client) {
    if (client == null) {
        throw new IllegalArgumentException("Client cannot be null!");
    }
    synchronized (mLock) {
        throwIfShutdownLocked();
        if (isConnectedLocked()) {
            throw new IllegalStateException("Already connected.");
        }
        mOwningUid = Binder.getCallingUid();
        registerUiTestAutomationServiceLocked(client);
        storeRotationStateLocked();
    }
}

private void registerUiTestAutomationServiceLocked(IAccessibilityServiceClient client) {
    IAccessibilityManager manager = IAccessibilityManager.Stub.asInterface(
            ServiceManager.getService(Context.ACCESSIBILITY_SERVICE));
    AccessibilityServiceInfo info = new AccessibilityServiceInfo();
    info.eventTypes = AccessibilityEvent.TYPES_ALL_MASK;
    info.feedbackType = AccessibilityServiceInfo.FEEDBACK_GENERIC;
    info.flags |= AccessibilityServiceInfo.FLAG_INCLUDE_NOT_IMPORTANT_VIEWS
            | AccessibilityServiceInfo.FLAG_REPORT_VIEW_IDS;
    info.setCapabilities(AccessibilityServiceInfo.CAPABILITY_CAN_RETRIEVE_WINDOW_CONTENT
            | AccessibilityServiceInfo.CAPABILITY_CAN_REQUEST_TOUCH_EXPLORATION
            | AccessibilityServiceInfo.CAPABILITY_CAN_REQUEST_ENHANCED_WEB_ACCESSIBILITY
            | AccessibilityServiceInfo.CAPABILITY_CAN_REQUEST_FILTER_KEY_EVENTS);
    try {
        // Calling out with a lock held is fine since if the system
        // process is gone the client calling in will be killed.
        manager.registerUiTestAutomationService(mToken, client, info);
        mClient = client;
    } catch (RemoteException re) {
        throw new IllegalStateException("Error while registering UiTestAutomationService.", re);
    }
}

UiAutomationConnection.connect函数接着调用了registerUiTestAutomationServiceLocked(client)，而registerUiTestAutomationServiceLocked主要是调用了IAccessibilityManager的registerUiTestAutomationService注册了一个Accessibility服务。

由此可见，UiAutomation最终也是使用了AccessibilityManagerService。

0x02 如何使用UiAutomation

UiAutomator的常见使用方式是调用uiautomator命令，或是将uiautomator.jar导入到自己的工程中。为了更加自由地使用UiAutomation提供的能力，可以考虑直接创建UiAutomation对象实例使用。由于UiAutomation的构造函数以及其它一些重要方法设置了@hide，因此无法直接使用，需要用反射的方式获取。

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
Object connection = null;
HandlerThread mHandlerThread = new HandlerThread("UiAutomationThread");
mHandlerThread.start();
try{
    Class<?> UiAutomationConnection = Class.forName("android.app.UiAutomationConnection");
    Constructor<?> newInstance = UiAutomationConnection.getDeclaredConstructor();
    newInstance.setAccessible(true);
    connection = newInstance.newInstance();
    Class<?> IUiAutomationConnection = Class.forName("android.app.IUiAutomationConnection");
    Constructor<?> newUiAutomation = UiAutomation.class.getDeclaredConstructor(Looper.class, IUiAutomationConnection);
    UiAutomation mUiAutomation = (UiAutomation)newUiAutomation.newInstance(mHandlerThread.getLooper(), connection);
    Method connect = UiAutomation.class.getDeclaredMethod("connect");
    connect.invoke(mUiAutomation);
    Log.i(TAG, ""+mUiAutomation);
    mUiAutomation.waitForIdle(1000, 1000 * 10);
    AccessibilityNodeInfo nodeInfo = mUiAutomation.getRootInActiveWindow();
    Log.i(TAG, ""+nodeInfo);
}catch(Exception e){
    e.printStackTrace();
    return;
}

这是一段最简单的使用代码，编译到apk里面，运行后报错了。

1
2
3
4
5
6
7
Caused by: java.lang.SecurityException: You do not have android.permission.RETRIEVE\_WINDOW_CONTENT required to call registerUiTestAutomationService from pid=3676, uid=10040  
 	at android.os.Parcel.readException(Parcel.java:1599)  
 	at android.os.Parcel.readException(Parcel.java:1552)  
 	at android.view.accessibility.IAccessibilityManager$Stub$Proxy.registerUiTestAutomationService  (IAccessibilityManager.java:352)  
 	at android.app.UiAutomationConnection.registerUiTestAutomationServiceLocked(UiAutomationConnection.java:337)  
 	at android.app.UiAutomationConnection.connect(UiAutomationConnection.java:89)  
 	at android.app.UiAutomation.connect(UiAutomation.java:197)

看来是需要android.permission.RETRIEVE_WINDOW_CONTENT权限，尝试将该权限添加到AndroidManifest.xml中，但是提示这个是系统权限，普通应用不能申请。因此，应用中不能使用UiAutomation的，同时发现，编译为jar包后使用shell权限执行，是可以正常跑过的。这也是为什么uiautomator工具可以正常运行的原因。

0x03 AccessibilityManagerService

如前所述，UiAutomation底层使用的还是AccessibilityManagerService，这里简单分析一下AccessibilityManagerService的实现。

该类内部包含了一个Service类：

1
2
3
4
5
6
7
8
9
/**
    * This class represents an accessibility service. It stores all per service
    * data required for the service management, provides API for starting/stopping the
    * service and is responsible for adding/removing the service in the data structures
    * for service management. The class also exposes configuration interface that is
    * passed to the service it represents as soon it is bound. It also serves as the
    * connection for the service.
    */
class Service extends IAccessibilityServiceConnection.Stub implements ServiceConnection, DeathRecipient;

从描述上看，每个Service实例代表了一个Accessibility服务，同时它实现了ServiceConnection接口，因此它也是一个Service客户端。

Service类中定义了一个bool类型的变量mIsAutomation，表示当前服务是否是UiAutomation。在AccessibilityManagerService的registerUiTestAutomationService方法中，将当前服务的组件名称设置为sFakeAccessibilityServiceComponentName，而mIsAutomation就是通过组件名称来判断是否是UiAutomation。

1
2
3
accessibilityServiceInfo.setComponentName(sFakeAccessibilityServiceComponentName);

mIsAutomation = (sFakeAccessibilityServiceComponentName.equals(componentName));

由于UiAutomation与AccessibilityService实现方式不同，因此，AccessibilityManagerService在很多地方都会根据mIsAutomation执行不同的逻辑。

根据前面的分析，在UiAutomation发起connect请求后，会进入AccessibilityManagerService的registerUiTestAutomationService。然后经过一系列的函数调用，进到Service的bindLocked函数。

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
/**
    * Binds to the accessibility service.
    *
    * @return True if binding is successful.
    */
public boolean bindLocked() {
    UserState userState = getUserStateLocked(mUserId);
    if (!mIsAutomation) {
        if (mService == null && mContext.bindServiceAsUser(
                mIntent, this,
                Context.BIND_AUTO_CREATE | Context.BIND_FOREGROUND_SERVICE_WHILE_AWAKE,
                new UserHandle(mUserId))) {
            userState.mBindingServices.add(mComponentName);
        }
    } else {
        userState.mBindingServices.add(mComponentName);
        mService = userState.mUiAutomationServiceClient.asBinder();
        mMainHandler.post(new Runnable() {
            @Override
            public void run() {
                // Simulate asynchronous connection since in onServiceConnected
                // we may modify the state data in case of an error but bind is
                // called while iterating over the data and bad things can happen.
                onServiceConnected(mComponentName, mService);
            }
        });
        userState.mUiAutomationService = this;
    }
    return false;
}

bindLocked会发送一个异步消息，调到onServiceConnected回调。

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
@Override
public void onServiceConnected(ComponentName componentName, IBinder service) {
    synchronized (mLock) {
        mService = service;
        mServiceInterface = IAccessibilityServiceClient.Stub.asInterface(service);
        UserState userState = getUserStateLocked(mUserId);
        addServiceLocked(this, userState);
        if (userState.mBindingServices.contains(mComponentName) || mWasConnectedAndDied) {
            userState.mBindingServices.remove(mComponentName);
            mWasConnectedAndDied = false;
            try {
                mServiceInterface.init(this, mId, mOverlayWindowToken);
                onUserStateChangedLocked(userState);
            } catch (RemoteException re) {
                Slog.w(LOG_TAG, "Error while setting connection for service: "
                        + service, re);
                binderDied();
            }
        } else {
            binderDied();
        }
    }
}

这里的mServiceInterface其实就是UiAutomation构造函数中实例化的mClient的远程对象。接着，调用了IAccessibilityServiceClient的init函数，该函数的实现位于AccessibilityService.IAccessibilityServiceClientWrapper类中。

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
public void init(IAccessibilityServiceConnection connection, int connectionId,
        IBinder windowToken) {
    Message message = mCaller.obtainMessageIOO(DO_INIT, connectionId,
            connection, windowToken);
    mCaller.sendMessage(message);
}

case DO_INIT: {
    mConnectionId = message.arg1;
    SomeArgs args = (SomeArgs) message.obj;
    IAccessibilityServiceConnection connection =
            (IAccessibilityServiceConnection) args.arg1;
    IBinder windowToken = (IBinder) args.arg2;
    args.recycle();
    if (connection != null) {
        AccessibilityInteractionClient.getInstance().addConnection(mConnectionId,
                connection);
        mCallback.init(mConnectionId, windowToken);
        mCallback.onServiceConnected();
    } else {
        AccessibilityInteractionClient.getInstance().removeConnection(
                mConnectionId);
        mConnectionId = AccessibilityInteractionClient.NO_ID;
        AccessibilityInteractionClient.getInstance().clearCache();
        mCallback.init(AccessibilityInteractionClient.NO_ID, null);
    }
} return;

这样，初始化完成后，AccessibilityInteractionClient实例中保存了IAccessibilityServiceConnection实例，而AccessibilityManagerService中对应的Service对象中也保存了IAccessibilityServiceClient实例，从而建立起双向的Binder通信。

在发生Accessibility事件后，AccessibilityManagerService会通过IAccessibilityServiceConnection的onAccessibilityEvent方法将事件通知给UiAutomation。

0x04 UiAutomator与AccessibilityService

AccessibilityService是一个继承自Service的抽象服务类，用户在使用时需要实现一个自己的子类。该类很大程度上依赖于AccessibilityInteractionClient类提供的接口，AccessibilityInteractionClient类内部保存了一个静态的LongSparseArray<AccessibilityInteractionClient> sClients对象，用来实现线程相关的单例对象。

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
/**
    * @return The client for the current thread.
    */
public static AccessibilityInteractionClient getInstance() {
    final long threadId = Thread.currentThread().getId();
    return getInstanceForThread(threadId);
}

/**
    * <strong>Note:</strong> We keep one instance per interrogating thread since
    * the instance contains state which can lead to undesired thread interleavings.
    * We do not have a thread local variable since other threads should be able to
    * look up the correct client knowing a thread id. See ViewRootImpl for details.
    *
    * @return The client for a given <code>threadId</code>.
    */
public static AccessibilityInteractionClient getInstanceForThread(long threadId) {
    synchronized (sStaticLock) {
        AccessibilityInteractionClient client = sClients.get(threadId);
        if (client == null) {
            client = new AccessibilityInteractionClient();
            sClients.put(threadId, client);
        }
        return client;
    }
}

前面提到，AccessibilityInteractionClient对象中保存了IAccessibilityServiceConnection实例，因此，可以调用该接口提供的功能。

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
/**
    * Interface given to an AccessibilitySerivce to talk to the AccessibilityManagerService.
    *
    * @hide
    */
interface IAccessibilityServiceConnection {

    void setServiceInfo(in AccessibilityServiceInfo info);

    boolean findAccessibilityNodeInfoByAccessibilityId(int accessibilityWindowId,
        long accessibilityNodeId, int interactionId,
        IAccessibilityInteractionConnectionCallback callback, int flags, long threadId);

    boolean findAccessibilityNodeInfosByText(int accessibilityWindowId, long accessibilityNodeId,
        String text, int interactionId, IAccessibilityInteractionConnectionCallback callback,
        long threadId);

    boolean findAccessibilityNodeInfosByViewId(int accessibilityWindowId,
        long accessibilityNodeId, String viewId, int interactionId,
        IAccessibilityInteractionConnectionCallback callback, long threadId);

    boolean findFocus(int accessibilityWindowId, long accessibilityNodeId, int focusType,
        int interactionId, IAccessibilityInteractionConnectionCallback callback, long threadId);

    boolean focusSearch(int accessibilityWindowId, long accessibilityNodeId, int direction,
        int interactionId, IAccessibilityInteractionConnectionCallback callback, long threadId);

    boolean performAccessibilityAction(int accessibilityWindowId, long accessibilityNodeId,
        int action, in Bundle arguments, int interactionId,
        IAccessibilityInteractionConnectionCallback callback, long threadId);

    AccessibilityWindowInfo getWindow(int windowId);

    List<AccessibilityWindowInfo> getWindows();

    AccessibilityServiceInfo getServiceInfo();

    boolean performGlobalAction(int action);

    oneway void setOnKeyEventResult(boolean handled, int sequence);
}

与UiAutomation不同的是，AccessibilityService的初始化是和普通的Service一致的。由于AccessibilityService比较特殊的地方在于需要在设置的辅助功能里开启对应的服务，点击开启后，会执行到BindService逻辑，进而执行到AccessibilityService的onBind回调，并触发AccessibilityManagerService中Service的onServiceConnected回调。

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
/**
    * Implement to return the implementation of the internal accessibility
    * service interface.
    */
@Override
public final IBinder onBind(Intent intent) {
    return new IAccessibilityServiceClientWrapper(this, getMainLooper(), new Callbacks() {
        @Override
        public void onServiceConnected() {
            AccessibilityService.this.onServiceConnected();
        }

        @Override
        public void onInterrupt() {
            AccessibilityService.this.onInterrupt();
        }

        @Override
        public void onAccessibilityEvent(AccessibilityEvent event) {
            AccessibilityService.this.onAccessibilityEvent(event);
        }

        @Override
        public void init(int connectionId, IBinder windowToken) {
            mConnectionId = connectionId;
            mWindowToken = windowToken;

            // The client may have already obtained the window manager, so
            // update the default token on whatever manager we gave them.
            final WindowManagerImpl wm = (WindowManagerImpl) getSystemService(WINDOW_SERVICE);
            wm.setDefaultToken(windowToken);
        }

        @Override
        public boolean onGesture(int gestureId) {
            return AccessibilityService.this.onGesture(gestureId);
        }

        @Override
        public boolean onKeyEvent(KeyEvent event) {
            return AccessibilityService.this.onKeyEvent(event);
        }
    });
}

相对于UiAutomation只能在shell环境中执行，AccessibilityService是可以运行在app环境中的，但是需要用户手动开启服务会略显麻烦。

0x05 总结

UiAutomator和AccessibilityService作为两种不同的实现形式，拥有各自的优缺点，这两年流行的抢红包工具基本也是基于这两种方式实现的。在自动化中使用它们也能起到一些辅助作用。