Using Input Pipelines In Your Android Game

Real-time Android games have a few threads to worry about. You can't block up the main UI thread because it's needed by the OS. If you block it, even for a second, your app will be deemed unresponsive and users will get the dreaded "Force Close or Wait" dialog. If your game doesn't do anything in a logic or main thread then you should be in the clear. If you do use a main thread for your game and run it as fast as it can go, as is the case with 2D games that draw to the canvas or 3D games that simply have a lot of game logic to process, you may want to think about how you're handling your input.

I had problems with the original Light Racer's input. I was using the example from Lunar Lander originally which simply synchronized the main loop and the input entry point. For some reason, even though in every Java app I've ever developed before that has the same code, the game would randomly not give the synchronized block to the UI thread that's waiting for it. The main loop would synchronize on the same mutex and release it over and over, even sleeping for a few MS to allow for other threads to get in there, but still the UI thread wouldn't do it. It was very strange and frustrating. After thinking hard about the problem, I decided to switch to input pipelines and all of my problems were solved.

How does an input pipeline work?

For our example, I'm going to use motion events since they can be intense.

Normally, the OS has an input event ready to go so it calls Activity.onMotionEvent() with the event. In a game activity that has a game thread running, you usually don't want to just interrupt the thread because there's no telling where it is in the processing and if you change the input on it mid-logic, you could really cause some problems. So, instead, it works like this:

doLoop() {
synchronized(mutex) {
process everything
}
sleep(2)
}

And the input comes in like this
doMotionEvent() {
synchronized(mutex) {
process motion event
}
}

In theory, this works fine. The problem I've seen on Android is that it doesn't work fine. If the main loop is really CPU intensive, the motion even sometimes gets stuck and is never able to get into the synchronized block! I don't know why that is. In a normal Java VM, the moment that the main loop is out of the synched block, the next thread that is waiting to get into the block is allowed to go.

An Input Pipeline is a simple a queue of input events that don't block the UI thread at all. Here's kind of how it looks:

The UI Thread calls:
doMotionEvent(MotionEvent e) {
InputEvent ie = convertToInputEvent(e);
gameThread.feedInput(ie);
}

We convert the MotionEvent into our own event and then feed it to the game thread.

On the game thread side, it needs to be able to handle the feed and then process the event.

feedInput(InputEvent ie) {
synchronized(inputQueueMutex) {
addToInputQueue(ie);
}
}

There's still a synchronized block but it's much more specific. This mutex is only used for 2 methods: feedInput and processInput. Guess what calls processInput? The main loop, and since it's for such a small fragment of time, feedInput is almost never blocked. This works much better than synchronizing the entire game loop which, for a really computationally intensive game, can and does get held for nearly 100% of the time.

Ready for the code? Here it is:

The InputObject:

public class InputObject {
public static final byte EVENT_TYPE_KEY = 1;
public static final byte EVENT_TYPE_TOUCH = 2;
public static final int ACTION_KEY_DOWN = 1;
public static final int ACTION_KEY_UP = 2;
public static final int ACTION_TOUCH_DOWN = 3;
public static final int ACTION_TOUCH_MOVE = 4;
public static final int ACTION_TOUCH_UP = 5;

public ArrayBlockingQueue<InputObject> pool;
public byte eventType;
public long time;
public int action;
public int keyCode;
public int x;
public int y;

public InputObject(ArrayBlockingQueue<InputObject> pool) {
this.pool = pool;
}

public void useEvent(KeyEvent event) {
eventType = EVENT_TYPE_KEY;
int a = event.getAction();
switch (a) {
case KeyEvent.ACTION_DOWN:
action = ACTION_KEY_DOWN;
break;
case KeyEvent.ACTION_UP:
action = ACTION_KEY_UP;
break;
default:
action = 0;
}
time = event.getEventTime();
keyCode = event.getKeyCode();
}

public void useEvent(MotionEvent event) {
eventType = EVENT_TYPE_TOUCH;
int a = event.getAction();
switch (a) {
case MotionEvent.ACTION_DOWN:
action = ACTION_TOUCH_DOWN;
break;
case MotionEvent.ACTION_MOVE:
action = ACTION_TOUCH_MOVE;
break;
case MotionEvent.ACTION_UP:
action = ACTION_TOUCH_UP;
break;
default:
action = 0;
}
time = event.getEventTime();
x = (int) event.getX();
y = (int) event.getY();
}

public void useEventHistory(MotionEvent event, int historyItem) {
eventType = EVENT_TYPE_TOUCH;
action = ACTION_TOUCH_MOVE;
time = event.getHistoricalEventTime(historyItem);
x = (int) event.getHistoricalX(historyItem);
y = (int) event.getHistoricalY(historyItem);
}

public void returnToPool() {
pool.add(this);
}
}

In the game activity:

private ArrayBlockingQueue<InputObject> inputObjectPool;

protected void onCreate(final Bundle savedInstanceState) {
super.onCreate(savedInstanceState);
//gameView = new GameView(this);
//setContentView(gameView);
createInputObjectPool();
}

private void createInputObjectPool() {
inputObjectPool = new ArrayBlockingQueue<InputObject>(INPUT_QUEUE_SIZE);
for (int i = 0; i < INPUT_QUEUE_SIZE; i++) {
inputObjectPool.add(new InputObject(inputObjectPool));
}
}

@Override
public boolean onTouchEvent(MotionEvent event) {
// we only care about down actions in this game.
try {
// history first
int hist = event.getHistorySize();
if (hist > 0) {
// add from oldest to newest
for (int i = 0; i < hist; i++) {
InputObject input = inputObjectPool.take();
input.useEventHistory(event, i);
gameThread.feedInput(input);
}
}
// current last
InputObject input = inputObjectPool.take();
input.useEvent(event);
gameThread.feedInput(input);
} catch (InterruptedException e) {
}
// don't allow more than 60 motion events per second
try {
Thread.sleep(16);
} catch (InterruptedException e) {
}
return true;
}

In the game thread:

private ArrayBlockingQueue<InputObject> inputQueue = new ArrayBlockingQueue<InputObject>(INPUT_QUEUE_SIZE);
private Object inputQueueMutex = new Object();

public void feedInput(InputObject input) {
synchronized(inputQueueMutex) {
try {
inputQueue.put(input);
} catch (InterruptedException e) {
Log.e(TAG, e.getMessage(), e);
}
}
}

private void processInput() {
synchronized(inputQueueMutex) {
ArrayBlockingQueue<InputObject> inputQueue = this.inputQueue;
while (!inputQueue.isEmpty()) {
try {
InputObject input = inputQueue.take();
if (input.eventType == InputObject.EVENT_TYPE_KEY) {
processKeyEvent(input);
} else if (input.eventType == InputObject.EVENT_TYPE_TOUCH) {
processMotionEvent(input);
}
input.returnToPool();
} catch (InterruptedException e) {
Log.e(TAG, e.getMessage(), e);
}
}
}
}

Then in your main loop, just call processInput().

This is what I use for all of my real-time games and it works very, very well. If you can't get it to work, spend some time looking at the concept and how the pools are used to reuse objects.

I hope this solves the mysterious Wait or Force Close problems for you. Even for games that I didn't have that problem on, I noticed that the input was handled better because it didn't skip over a few ticks waiting to get the synchronized block.

Enjoy!

43 Comments

Post a comment here or discuss this and other topics in the forums

Thanks for all the help (as I

Thanks for all the help (as I stated in the previous comment :) ). Still have another question which I've put in the forums, under the name Themuzz.

Thanks

Thanks man, for all the help. Your right that I do some not very optimal things in the code :) Just wanted to port an old game I made to this new template.

But I think your right, that it's not possible to make it faster... But I have to say it's definitely a lot faster than the old way I used.

So thanks again! And if I find someway to make it faster, I'll post it (in the fourms :) ).

I just tried out the code. I

I just tried out the code. I think it works fairly well. While there may be a way to make it more responsive, I'm not aware of it. You could maybe ignore historical touch movement, but I'm not sure that that is your problem. In fact, I don't think you have a problem. My guess is that this is good as you're going to be able to get it. Let me know if you do find a way to make it faster, though. I'm always interested in hearing about new optimizations.

I looked at your code. It

I looked at your code. It mostly looks sound. I see something that makes me think it's an input processing problem, though

I see when you're processing the motion events, you only do the logic for action 3, which should be the constant InputObject.ACTION_TOUCH_DOWN. I noticed that there is no code to handle action 4, which is InputObject.ACTION_TOUCH_MOVE, though you implicitly handle it because on_clicked should still be true while there is input to process and on_clicked hasn't been set to false by the ACTION_TOUCH_UP clause.

Now that I've said that out loud, I'm realizing that it looks ok, too, though it's not exactly the most straightforward way of writing it :)

Really, that should be responsive. I noticed that you don't explicitly clear the screen but instead rely on a call to _panel.onDraw(Canvas) which only does canvas.drawColor(0xFFCCCCCC); so that should be fine...

Normally I'd give you some suggestions and let you work on them but I'm curious as to what the source of the problem is so I'll set up the project and try it out.

By the way, would you mind posting a follow up in the forum? I'd rather carry on the conversation there.

Thanks!

Thanks, works! One little question...

Thanks for all the help, it works now!

But I still have one question, and I'm kinda afraid to ask it since I've received so much help from you.
The problem is that I have a ball on the screen which can be dragged around. The only problem is that the ball is still a little slower than the finger, so if one draws circles the ball will follow on quite an end behind of the finger.

First I thought this was because of the low framerate of the view I used first, but the SurfaceView has the same problem. If I look in the logs which display the coordinates of the moving finger I see that they also are behind, so it's not my code that uses the coordinates to draw the ball.

So I think I'm doing something wrong since I tried to make the Thread sleep longer and shorter, but both don't work. I removed all my code and it's still slow in the output. If I look at the game activity onTouchEvent, it shows the coordinates realtime, so somewhere in the InputObject it gets slowed down. I tried really a lot today and yesterday, but I can't get it to work correctly.

So I'm asking whether or not you would like to look at my code to help me. If not, just say so and I'll understand.

Code:
http://www.todaypda.com/a-test_new.zip

Thanks again, and if there is anything I can do for you, just let me know!

(if you want, you may post the code on your website as an example)

Oops

I forgot to add this code snippet. I call this from onCreate(). I'll update the example to reflect these changes.

private void createInputObjectPool() {
inputObjectPool = new ArrayBlockingQueue<InputObject>(INPUT_QUEUE_SIZE);
for (int i = 0; i < INPUT_QUEUE_SIZE; i++) {
inputObjectPool.add(new InputObject(inputObjectPool));
}
}

Thanks!

Thanks for the help! I really appreciate it!

I'm still testing your code in a test project and have a little problem. When there is a touchevent the following code will be called:


InputObject input = inputObjectPool.take();
input.useEvent(event);
gameThread.feedInput(input);

But the processing will stop at the first line, since the inputObjectPool is empty and the take() will wait if no elements are present on this queue. So I tried to find out when an element gets put to the queue and that happens here:


public void feedInput(InputObject input) {
synchronized(inputQueueMutex) {
try {
inputQueue.put(input);
} catch (InterruptedException e) {
Log.e(TAG, e.getMessage(), e);
}
}
}

But this part only gets called when the code in the beginning has been parsed, but that code stops at the first line and doesn't get to the third.

Perhaps I'm missing something? I'm really busy to get this to work, but I'm out of luck.

Thanks for all the help so far!

ps I will use the structure you recommend, thanks for the tip!

I'll update my code example

I'll update my code example but you'll notice that the inputObjectPool was declared but never instantiated, which is why you get a null pointer exception. All you need to do is create a new one in your constructor.

I had a quick look at your code, can I recommend that you use the following structure?

GameActivity extends Activity
GameView extends SurfaceView
GameThread extends Thread

If the game is full screen, don't use a layout file. Just use a surface view and create the view directly, then set it as your content view with setContentView(gameView). Have the view create your thread and just have your activity talk to your view, telling it to pause/resume, etc, which will in turn tell the thread to pause/resume, etc. That's the easiest way to manage it for a simple game.

Fast reply!

Thanks for the fast reply!

The thing is, I'm an average java programmer and I'm not very home in stopping threads etc. I can create everything what I want, but my code is never optimal, and I would never think of things like this (thank you again for it). I found out that a simple game I created with some balls get's very slow when someone touches the screen and the input is handled.

So I tried a new project with your code, but I think I misunderstand what you mean with the game thread and game activity. So if I'm not asking too much, would you have a look at it? I understand if you only want to give hints, perhaps I'm just missing something pretty important.

I've uploaded my whole test project here:
http://www.todaypda.com/a-test_all.zip
Or only the java code:
http://www.todaypda.com/a-test_code.zip
(at the moment I get a nullpointer exception in Main.java at this: "InputObject input = inputObjectPool.take();", like the ArrayBlockingQueue is empty)

If you don't want to look at it, I'll understand, just let me know :) I wanted to paste the code here but I don't think you would like that :)

Thanks again for the nice tutorial, to bad it's a little to advanced for me :D

ps. If you don't like the links in the comment, just remove it. And if you prefer mail to respond instead of the comments I'll understand.

Thanks, Sander! I'd love to

Thanks, Sander! I'd love to help and post more code but I left it open at processKeyEvent() and processMotionEvent() because those become totally game-specific. Where are you having problems? If you get more specific, I can try to help.

Nice!

Hello Robert,
Thanks for posting this! I'm now two weeks busy with fixing this issue with ontouch events, and now I found your code. Only one thing, I can't get it to work :) If I'm not asking too much, could you post an example? (just with this code implemented, not with a game code or anything).

Thank you again!

Thanks for letting me know.

Thanks for letting me know. I've benefited so much from other sharing what they've discovered that I feel it's the least I can do. I'm happy to help!

Wow! Thank you!

I'm glad you post findings like this, as it helps me in my own game. You've already done so without knowing it on numerous occasions, and I just wanted to say thank you.