Modules - Leveraging ARTist’s capabilities

ARTist is designed to be flexible enough to cover a large field of use cases. It does so by exposing a Module interface for developers so they can focus on implementing interesting analyses and transformations on the targets, while ARTist does all the heavy lifting in the background. In fact, there are countless possibilities on what to implement with ARTist modules. The rule of thumb is: You get full access to the target’s Java code and you can tamper with it as much as you like. Typical use cases include the hooking of methods, adding logging and debugging helpers, running lightweight static analyses, changing configurations or even patching vulnerabilities. The Module’s functionality is completely up to you and we provide a growing set of convenience methods and boilerplate code to ease the process as much as possible.

If you want to learn how to write modules, proceed here. If you want to understand the design and principles underlying ARTist modules, you can have a look at this page.

For either case, we recommend to have a look at this design document since it gives a general introduction in what we think modules are and what they should and should not do.

Current Design

We see a Module as the implementation of a certain use case, a well-defined kind of change in behavior and appearance, or a particular analysis. The Module is a logical, self contained unit that should by itself be usable on any other ARTist installation of any user. This-self containment implies that we do not have strong inter-dependencies between modules. If you have two algorithms that modify the target’s code but only work when applied together, they belong on the same Module. If, however, the functionalities are compatible and can be combined, but do not depend on each other, it can be two modules as well.

Currently, there are two parts of an ARTist Module from a technical perspective: The compiler side and the library side.

The Compiler Side

The Android Runtime’s dex2oat compiler was designed to support state-of-the-art optimizations to make the execution of compiled apps as smooth as possible. It is important to understand some basic principles the compiler follows in order to understand the way we integrated ARTist and develop modules for it.

Dex2oat does a great job in scaling with parallellization since the whole process of transforming app’s bytecode into the compiler IR, running optimization passes on it and then generating native code is done on a per-method level. This in particular implies that one thread only sees the IR of one target method at the time and applies the optimizations on exactly this level. It is paramount to keep this in mind: When compiling methods, there is just no holistic view of the compilation target (app or framework code) at this point. The reason that this implementation detail is so significant is explained by the simple fact that ARTist makes heavy use of dex2oat’s optimization framework. From the compiler’s view, ARTist modules are just providers of additional optimization passes that will be executed along with the other ones. And this is already half of the story: We implement our modifications and customizations in specialized optimization passes that we call ARTist passes. Hence, your code sees exactly one method at a time. This does not mean you cannot keep a state between executions of your ARTist passes, but it has a large impact on how we have to design modules. Now equipped with the knowledge that ARTist passes essentially are optimization passes, it is easy to understand why they integrate so nicely with dex2oat. The optimization framework does not make too much assumptions about optimizations. It just collects a list of available optimizations and then executes them one after another while granting them full access to the method that is currently under compilation. Since many optimizations already change the code a lot, we can do so as well. While in theory, we could place our modules anywhere in this queue, it is quite useful to apply ARTist passes after all other optimizations have been executed to avoid hard-to-debug interferences, such as for example a reference monitor call being moved around for optimization reasons.

The second half of the story bridges the gap between what we came to know as ARTist passes and the compiler’s side of an ARTist Module. When ARTist passes implement the business logic, what does the Module do then? It represents a semantic unit by bundling all ARTist passes that are required to implement a certain use case, adds filtering functionality to decide when to use which pass, defines metadata, etc. In general, it collects the information required for ARTist to decide what to execute and when.

The Java Library Side

While a compiler optimization pass gives us full access to a target’s method and, in theory, we are then capable of applying arbitrary changes, most of the time it is way too cumbersome to create complex functionality in the compiler’s IR. Also, if we are hooking methods, we have a strict coupling of the hooking code and the new methods where our hooks redirect to. The solution to this is quite simple: Instead of implementing a lot of functionality in the compiler, we simply write a Java library with all the methods we need and in our ARTist pass we just add method calls into this library. Let’s make an example. One of the default modules is the trace Module that modifies methods to print their own name to logcat as soon as they are called. We could have created the reflection code that inspects the current call stack and pulls out the correct method information in the ARTist pass, but this is a lot of work and also incurs overhead when updating the code that shall be injected. A smarter way of implementing this is to write a companion Java library we call Codelib that encapsulates all the required functionality to find a caller’s method name. Now, all we have to do in the ARTist pass is to inject a call to the Codelib method that implements the tracing functionality. The result is a very simple ARTist pass and nice library where we can implement and update our call targets with Java. The nice thing about this decision is that you have a cleaner, decoupled design and can implement your functionality faster and with higher maintainability. Of course, there are also downsides, because there is only a limited number of method IDs available per .dex file.

Putting it Together

Now that we know about both sides, the compiler’s and the library’s, we can briefly talk about how this works together. As explained above, having the more complex functionality in the Codelib allows for a cleaner design and an improved workflow for Module developers. The result is that what we consider an ARTist Module right now is essentialle the following: A compiler module that bundles ARTist passes, a compiled Java library and some additinal files and metadata.

So how does it currently work together: Assume you want to apply your newly created ARTist module to an app. First, you need to make sure the Codelib’s methods are available. This is where we use our Dexterous tool to add the Codelib to an intermediate app package where we register the new method IDs. Second, ARTist is used to modify the app by recompiling the intermediate app package that is aware of our Codelib. During compilation, your Module can inject calls to your Codelib library methods because they have been registered before. The resulting compiled version of the app contains the complete Codelib code and all the changes applied by your Module. We now tell Android to use our new version of the compiled app instead the old one and that’s it. Depending on whether you deploy ARTist as an app or use it in an AOSP build, the steps are implemented a bit differently, but the structure stays the same.

Caution! We use the term Module for the whole ARTist module that includes the copiler and library part, but also for the compiler part that bundles ARTist passes, filters, etc. We will refine this in the future, but for the time being, make sure to pay attention to the context for disambiguation.

The Road Ahead

Providing developers with a powerful, yet flexible solution for Android app and platform analysis and modification has been the goal for the ARTist project from the very beginning. However, we just entered the beta phase so there is a lot to do.

Eventually, we would like to provide a hub or market place for modules where developers can share their creations. Think Xposed modules for the ARTist framework.

In conclusion, ARTist is quite young but we are actively working on the ecosystem and keep improving things, and one of the core features we want to have are independent modules written against our open module SDK by members from the community.