Note: input from the lower level kernel
Note: This file is a sample chapter from the full book – “Android Internals: A confectioner’s cookbook” – which can be found on http://NewAndroidBook.
com/. The chapter was made available for free as a preview of the book (think of it like Amazon’s “Look Inside” :-). I encourage you to check out Technologeeks.com Android Internals training, which builds on the book and expands it further with Instructor Led Training. You can also preorder the book by emailing [email protected] Note some links (to other chapters in the book) will not work in this file (since it is partial), but external links will.
Feedback, questions and requests are always welcome.Chapter XII – The Android Input ArchitectureMost users interact with their device through its touch screen. Android, however, can handle input from multiple input sources, of which the touch screen is only one.
Additional sources may be a physical keyboard, a mouse, joystick or other controller, and virtually any other HID (Human Interface Devices), or even device sensors. Android provides a surprisingly elaborate stack structure, which receives input from the lower level kernel driver, translates the raw input data into a more manageable higher level event (key press, gesture, and such), and then propagates that to the active view in the foreground activity, by (eventually) invoking the callback associated with the view. On the way, Android uses both native level and Java level components. The high level view is shown in figure 11-1:Figure 11-1:The Android Input Stack Android: DalvikAndroid: Native Linux KernelInput Device DriverInterrupt Linux Input Stack /dev/input/eventXXApplication ViewWindowInputEventReceiverandroid.view.InputChannelInputChannelcom.
InputManagerService NativeInputManagerInputDispatcherPolicyInputReaderPolicyInputManagerEventHubInputReaderInputDispatcher This chapter explores the Android input stack. We’ll take a bottom-up approach, starting with the lower layer – provided by the device driver and the Linux kernel, through the runtime frameworks, conversion to an event, and finding the correct handler for that event. Be warned: The journey up the stack is long and arduous.
But understanding it will hopefully make you gain new appreciation to just what happens when you next touch your device.In This Chapter: The Linux Kernel Layer Interrupt Handling Kernel Input Stack The Native Layer InputManagerEventHubInputReaderInputDispatcherInputFlinger (Android L) The Dalvik Layer The Input PipelineThe Application View Debugging the Input Stack Input Handling at the Linux Kernel Layer Interrupt HandlingAn input event begins with an interrupt, which is generated when the device (touch screen, physical button, headphone jack, etc) detects the physical event. At the lowest level, this event causes electrical current, which is converted to an interrupt, which is delivered via the interrupt controller to the CPU, on one of the many interrupt request lines (IRQs).Linux provides a default interrupt handler in the form of do_IRQ. This function checks if any device drivers have “claimed” the interrupt (by calling request_irq()). Usually, each driver will claim the IRQ line corresponding to its device, though it is not uncommon to see interrupt lines shared between two or more drivers. A driver claiming an IRQ also supplies a callback – often referred to as an ISR (Interrupt Service Routine), which will be invoked by do_IRQ when an interrupt is received on the claimed IRQ.You can view the interrupt lines claimed by drivers if you look at /proc/interrupts.
This highly architecture dependent file will show you the interrupt count for interrupts which have been registered (a different file, /proc/stat, will show all interrupt counts). Each interrupt line is shown with the count, controller type, and registered driver name. The driver names are often quite cryptic, but in some cases (such as the Galaxy S series) are well defined. The Galaxy S4 also shows a good example of multiple interrupt controllers, as shown in the following output, demonstrating some of the more interesting interrupts on the device:[email protected]:/ $ cat /proc/interrupts CPU0 17: 0 GIC dg_timer 18: 9245245 GIC gp_timer # Timer interrupt … 34: 1 GIC MSM_L2 36: 243472 GIC synaptics_rmi4_i2c # Touch screen 47: 0 GIC pm8821_sec_irq 48: 981035 GIC msmgpio .
..134: 4960502 GIC msm-sdcc (cmd), msm-sdcc (pio) #135: 950126 GIC msm-sdcc (cmd), msm-sdcc (pio) # Flash storage136: 4359946 GIC msm-sdcc (cmd), msm-sdcc (pio) # …343: 107 msmgpio max77693-irq351: 517 msmgpio bluetooth hostwake # Bluetooth devices353: 503146 msmgpio bcmsdh_sdmmc362: 453332 msmgpio pm8xxx_usr_irq365: 0 msmgpio sii8240369: 0 msmgpio sec_headset_detect # headset jack371: 0 msmgpio msm_hsl_wakeup474: 0 pm8xxx pm8921_overtemp_irq479: 3896 pm8xxx pm8xxx_rtc_alarm486: 0 pm8xxx pm8921_batt_alarm_irq489: 0 pm8xxx msm_otg490: 63 pm8xxx pmic8xxx_pwrkey_release # Power down491: 63 pm8xxx pmic8xxx_pwrkey_press # Power up ..
.661: 579 pm8xxx home_key # home button662: 1338 pm8xxx cypress_touchkey # menu, back664: 0 pm8xxx msm-sdcc (slot)666: 28 pm8xxx volume_up_key # Volume UP 668: 56 pm8xxx volume_down_key # Volume DOWN ..
.# Inter processor (core) interrupts: Note four columns for quad-coreIPI0: 0 13962 13870 11649 CPU start interruptsIPI1: 0 0 0 0 Timer broadcast interruptsIPI2: 3947099 5539984 2449051 1427641 Rescheduling interruptsIPI3: 1369 12875 12846 12533 Function call interruptsIPI4: 402 126466 109265 91618 Single function call interruptsIPI5: 0 0 0 0 CPU stop interruptsIPI6: 0 0 0 0 CPU backtraceErr: 0 In other cases, however, like the HTC One M8, the physical keys are all grouped under one interrupt, gpio_keys. Another interesting feature is that on multi-core CPUs, you will actually see multiple interrupt count columns when additional cores are active. You can further control which core responds to which interrupt by writing a hexadecimal mask value to the /proc/irq/##/smp_affinity. Advanced boards may also show “IPI” (inter-processor interrupts) as well.
You can empirically determine which IRQs are associated with which devices by viewing /proc/interrupts before and after triggering an interrupt from a device (for example, touching the screen, or pressing a button). A physical button press will usually result in 2 or more interrupts (corresponding to the button down and button up events), whereas swipes and other gestures may very well result in dozens or more, the same as mouse motion would on a desktop system.Interrupts are effectively handled at the highest possible priority. With rare exceptions, an interrupt will preempt whichever thread is executing on the CPU, as it demands the kernel’s immediate attention. Desktop systems make use of this fact in some cases (for example, with the SysRQ mechanism), though a more immediate application is that, when a device is hanging, most users intuitively press buttons, tap, swipe, or otherwise try to “wake up” the device. In practice, this can, in theory, be detrimental, as more events trigger more interrupts, which can increase the system load by preempting what the CPU was doing, in favor of handling otherwise meaningless gestures.
In practice, however, interrupt handling is so quick the load they introduce is fairly minimal.Well behaved drivers obey the strict requirements of handling IRQs – they never block, and take as little time as possible, deferring work toa software IRQ (the kernel’s ksoftirq threads). Interrupts therefore usually have minimal effects on the system.
In addition some devices can use tricks such as interrupt coalescing (firing one interrupt instead of several), and devices will not fire interrupts when the device is sleeping (for example, the touch screen, when the display is turned off). The Linux Input Driver ModelAndroid uses the standard Linux input driver model, introduced back in 2.4 and standardized in 2.6 and later. The Linux kernel documentation contains a subdirectory documenting the model and usage of its programming interfaces (Documentation/input/, and in particular input.txt).
The interested reader is encouraged to consult the documentation for more detail, though we provide an overview in this section.A device driver responsible for an input device is required to allocate an input_dev structure and populate its capabilties. This structure is defined in
.)mscbitmiscellaneous events supported by deviceEVIOCGBIT(EV_MSC..)ledbitLEDs present on the deviceEVIOCGBIT(EV_LED.
.)sndbitsound effects supported by deviceEVIOCGBIT(EV_SND..)ffbitsupported force feedback effects, if anyEVIOCGBIT(EV_FF..)swbitswitches present on the deviceEVIOCGBIT(EV_SW.
.)hint_events_per_packetaverage # of events generated by devicekeycodemax size of keycode tablekeycodesize size of elements in keycode tablekeycodemap of scancodes to keycodes for devicegetkeycode(legacy) retrieve current keymap. ffForce-Feedback, if any repeat_keyLast pressed key, for auto-repeat timerauto-repeat timer repauto-repeat parameters mtstruct input_mt holding Multitouch state absinfoAbsolute axes coordinate information keycurrent state of device keys/buttonsEVIOCGKEY ledcurrent state of device LEDs, if anyEVIOCGLED swcurrent state of device switches, if anyEVIOCGSW opencallback for open(2) on device closecallback for close(2) on device flushflush device events,e.g. force-feedback eventhandler for events sent to deviceFigure figInputDev: The struct input_dev (from
g. event0 would have minor 64, event1 would have minor 65, etc). Operations on the created device node will be handled by an evdev_fops file_operations structure (in
As shown in the figure, the key fields of the input_dev structure are accessible via standard ioctl(2) calls with specific EVIOC* constants. Additionally, the capabilties and properties of a device are exported to user mode via entries in sysfs, under /sys/class/input/event##/device/capabilities/ and /sys/class/input/event##/device/properties.The Linux device drivers respond to interrupts, generated by the respective devices. The drivers then report the events using the input_report_key/… functions, and events are then queued onto the /dev/input/event## device as structs containing the timestamp, event type, associated code and value. User mode applications use the standard system calls (that is read(2), select(2)/poll(2) and the like) to retrieve events (always an integer multiple of sizeof(input_event)) from the device.
The supported event types are defined in and shown in table 11-evttypes: #Event codeSpecifies 0x00EV_SYNSeparate/synchronize other events (e.g. SYN_REPORT/SYN_MT_REPORT), or report events lost (SYN_DROPPED) 0x01EV_KEYKey press (KEY_*) or touch (BTN_TOUCH) 0x02EV_RELRelative changes to a property. Changes relayed through REL_XYZ values. 0x03EV_ABSAbsolute coordinates for an event.
Values are usually ABS_XYZ, or ABS_MT for multi-touch 0x04EV_MSCMiscellaneous codes 0x05EV_SWBinary switches. E.g. SW_JACK_PHYSICAL_INSERT for headphone insertion 0x11EV_LEDUsed for device LEDs, if any 0x12EV_SNDUsed for sound devices 0x14EV_REPUsed for auto-repeating events 0x15EV_FFUsed for force-feedback capable devices (e.g. joysticks).
An EVIOCSFF ioctl may be used to upload force feedback effects 0x16EV_PWRReserved for power events. Largely unused 0x17EV_FF_STATUSUsed for force-feedback capable devices. Naturally, not all input devices support all event classes; The input_dev structure maps the various events a particular device supports by its bitmaps, and the device’s sysfs entry makes those bitmaps visible in user mode as /sys/class/input/inputXX/capabilities. The input event source of each device is in /sys/class/input/inputXX/eventXX, and also conveniently symlinked directly from /sys/class/input. Note, that even though the Linux input model was designed for HID type devices, it can be used on virtually any devices, including sensors. This is shown in the following experiment. Experiment: Looking at low-level input eventsYou can examine input events yourself through the shell.
First, look at the files in /dev/input. On the emulator, you should see something like this:[email protected]:/ # ls -l /dev/inputcrw-rw—- root input 13, 64 2013-11-15 18:24 event0crw-rw—- root input 13, 63 2013-11-15 18:24 micecrw-rw—- root input 13, 32 2013-11-15 18:24 mouse0Output on a real device will likely be different, as these have many more input channels, mapped to its various sensors; The Samsung S3, for example, has event0 through event13, and no “mouse0”. The major and minor numbers, however, should be the same, and the major (13) is associated with the kernel input driver subsystems (as can be verified with grep input /proc/devices).
The system keeps track of all devices quite conveniently in /proc/bus/input/devices:[email protected]_m8wl:/ $ cat /proc/bus/input/devices I: Bus=0000 Vendor=0000 Product=0000 Version=0000N: Name=”h2w headset”P: Phys=S: Sysfs=/devices/virtual/input/input6U: Uniq=H: Handlers=kbd event6 keychord B: PROP=0B: EV=3B: KEY=84 10000 138 0 e0800 0 0 0..To see the properties and capabilities of a given device, say /dev/input/event0, examine its sysfs entry:## File sizes are reported as 4k (pagesize) arbitrarily – all virtual [email protected]:/sys/class/input/event0$ ls -l-r–r–r– root root 4096 2013-11-15 18:24 devlrwxrwxrwx root root 2013-11-15 18:24 device -> ..
/../input0drwxr-xr-x root root 2013-11-15 18:24 powerlrwxrwxrwx root root 2013-11-15 18:24 subsystem -> ../../.
/class/input-rw-r–r– root root 4096 2013-11-15 18:24 [email protected]:/sys/class/input/event0 # cat ueventMAJOR=13MINOR=64DEVNAME=input/event0By accessing the device nodes, you can treat the input sources as files, and by using a command such as “cat” on them, dump raw events directly to standard output, though those would be appear to be nothing more than synchronized garbage. A better way would be to use the getevent command, which is part of toolbox. The source for this tool is at system/core/toolbox/getevent.c.
When used as getevent, toolbox invokes getevent_main(), which uses Linux’s inotify to enumerate the files in /dev/input/event. It then opens each detected device, and uses the ioctls from the previous table to obtain the device information. Finally, it adds the device to an array of file descriptors which it polls continuously. As each device (event source) signals input, getevent reads the event records and dumps it to stdout.getevent has several particularly useful switches: -i: Show HID codes and other information -l: Show numbers in human readable constant form (e.g. “0001” as KEY_ESC) -p: Show device capabilities (as per sysfs capabilities file) and exit -t: Show timestamps -h: Show help on all switches (-n, -t, -s, -S, -v, -d, -p, -i, -l, -q, -c and -r) then exit Experiment: Looking at low-level input events (cont.
) Armed with this information, you can conduct this simple experiment: Make sure your display isn’t sleeping, then run getevent -l via adb on your device (or emulator), and then touch the screen. On the emulator, you should see output similar to the following:[email protected]:/$ getevent -lcould not get driver version for /dev/input/mouse0, Not a typewriteradd device 1: /dev/input/event0 name: “qwerty2″could not get driver version for /dev/input/mice, Not a typewriter/dev/input/event0: EV_ABS ABS_X 00000083 # X position of touch/dev/input/event0: EV_ABS ABS_Y 00000129 # Y position of touch/dev/input/event0: EV_KEY BTN_TOUCH DOWN # Touch start /dev/input/event0: EV_SYN SYN_REPORT 00000000 # Event “terminator”/dev/input/event0: EV_KEY BTN_TOUCH UP # Touch end/dev/input/event0: EV_SYN SYN_REPORT 00000000 # Event “terminator” On a real device (for example, the Samsung S3) you’ll see much more output, owing to the myriad input sources it has. The msm8960 sensors, for example, are what enables headphone vendors to control music playing and call answering through the headphone jack.
[email protected]:/$ getevent -l## getevent will display all enumerated devices:add device 1: /dev/input/event6 name: “sec_touchscreen”add device 2: /dev/input/event8 name: “barometer_sensor”add device 3: /dev/input/event4 name: “sec_jack”add device 4: /dev/input/event3 name: “light_sensor”add device 5: /dev/input/event2 name: “proximity_sensor”add device 6: /dev/input/event0 name: “sii9234_rcp”add device 7: /dev/input/event12 name: “msm8960-snd-card Headset Jack”add device 8: /dev/input/event11 name: “msm8960-snd-card Button Jack”add device 9: /dev/input/event10 name: “msm8960-snd-card Volumeup Jack”add device 10: /dev/input/event9 name: “msm8960-snd-card Volumedown Jack”add device 11: /dev/input/event13 name: “sec_touchkey”add device 12: /dev/input/event1 name: “fsa9485” add device 13: /dev/input/event5 name: “sec_keys” # physical keysadd device 14: /dev/input/event7 name: “sec_powerkey” # power button – KEY_POWER# These are sent regularly from the light sensor, as it’s highly sensitive/dev/input/event3: EV_REL REL_X 00000020 /dev/input/event3: EV_REL REL_Y 00000021 /dev/input/event3: EV_REL REL_Z 00000015 /dev/input/event3: EV_REL REL_MISC 00000024 /dev/input/event3: EV_SYN SYN_REPORT 00000000 ## Touch event: Notice ABS_MT is used here, rather than BTN_TOUCH/dev/input/event6: EV_ABS ABS_MT_TRACKING_ID 00000043 # unique ID for this touch /dev/input/event6: EV_ABS ABS_MT_WIDTH_MAJOR 0000000a # major axis of approaching ellipse /dev/input/event6: EV_ABS ABS_MT_POSITION_X 0000011f # center x touch position /dev/input/event6: EV_ABS ABS_MT_POSITION_Y 0000022d # center y touch position/dev/input/event6: EV_ABS ABS_MT_TOUCH_MAJOR 00000010 # major axis of touch ellipse/dev/input/event6: EV_ABS ABS_MT_TOUCH_MINOR 0000000c # minor axis of touch ellipse/dev/input/event6: EV_ABS ABS_MT_TOOL_X ffffffc4 # center y tool position## Physical buttons: HOME and VOLUMEDOWN together (notice /dev/input/event5)/dev/input/event5: EV_KEY KEY_VOLUMEDOWN DOWN /dev/input/event5: EV_SYN SYN_REPORT 00000000 /dev/input/event5: EV_KEY KEY_HOMEPAGE DOWN /dev/input/event5: EV_SYN SYN_REPORT 00000000 /dev/input/event5: EV_KEY KEY_VOLUMEDOWN UP /dev/input/event5: EV_SYN SYN_REPORT 00000000 /dev/input/event5: EV_KEY KEY_HOMEPAGE UP/dev/input/event5: EV_SYN SYN_REPORT 00000000 Experiment: Simulating low-level events with sendeventAs it so happens, toolbox also has a sendevent tool which you can use to simulate events at the lowest level, by writing directly to the /dev/input/eventXX device nodes. The code itself is straightforward, constructing an input_event from the command line, and calling write(2) to send it to the device node. The caller needs write access to the device node – which the shell provides for you even on non-rooted device thanks to its group membership in the input group.Using this tool alongside getevent brings out the real usefulness of low-level UI Automation: With getevent, you can record input events – touch, swipe, and physical button events on most devices, but even sensor readings like orientation, proximity and light (on those devices which support it, like the Samsung S3, above).
You can then replay them with sendevent. There are only two caveats to remember: While getevent spits out the events in hexadecimal, sendevent will expect decimal input. So you will have to do the conversion yourself (or use a simple script to do so)Most events, though atomic to the user, are broken up into multiple components, and require the SYN_REPORT synthetic event to act as a delimiter. It’s therefore important to send that event at the end of the sequence.
The following example shows a simulation of pressing the home button on an S3. since the EV_KEY constants are the same across Linux (and Android) versions, the only adaptation required for other devices would be the figuring out which /dev/input/eventXX node to use. Continuing the previous output, we would have something like:# simulate EV_KEY KEYHOMEPAGE DOWN followed by [email protected]$ sendevent /dev/input/event5 1 172 1; sendevent /dev/input5 0 0 0# To simulate home button hold, delay the following line, simulating the UP/[email protected]$ sendevent /dev/input/event5 1 172 0; sendevent /dev/input5 0 0 0 As we continue traversing up the input stack, the next experiment you encounter will introduce you to yet another method of simulating input events. Input Handling at the Android Native Layer… (not part of this preview).