Touch interactions in game development for Windows Phone 7 Part 2 Multitouch

Programming! - 2.Try multi-touch

About this sample

Here, I would like to explain the program for realizing multi-touch input. In Windows Phone 7, multi-touch input via touch panel is one of the most important items for user input. This is because, unlike Windows and Xbox 360, the main input interface is the "touch panel". There may be Windows phones with keyboards, but of course they don't always have them. So, in order for the game to work on any Windows Phone, it needs to be touch-capable.

If you limit the touch panel to "single touch" only, the types of games that can be made will inevitably be limited when it comes to making games. Think of a handheld game console (you can leave it stationary). Most game consoles require you to hold the game controller (or actual console) with both hands and press several buttons at the same time.

Windows Phone 7 doesn't have as many buttons as game controllers, and it doesn't necessarily have a keyboard. Therefore, virtual keys and buttons may be placed on the screen, and when pressing them, it is inconvenient because they cannot be pressed simultaneously with only a single touch.

Of course, in addition to the virtual button example above, multi-touch is necessary for games where multiple people operate one screen, such as "pinch and stretch (move two points closer or farther apart)" that is common in multi-touch.

By the way, the introduction is long, but in this sample, I would like to acquire multi-touch information using a class dedicated to touch panels. It's fun to create a sample that can be played with multi-touch suddenly, but first I would like to find out what kind of information can be obtained with multi-touch. There are some habits, so by knowing them first, you can reduce the time spent investigating the cause, such as when it does not work well in programming later.

Goals of this sample program

Display and confirm the information when multi-touch is performed.

Figure 1: Touch information is displayed as text when touched

Program - Declaring Fields

The fields must be labeled TouchPanelCapabilities Structure and TouchCollection Structure.

/// <summary>
///  スプライトでテキストを描画するためのフォント
/// </summary>
SpriteFont font;

/// <summary>
///  タッチパネルの機能情報
/// </summary>
TouchPanelCapabilities capabilities;

/// <summary>
///  取得したタッチ情報の一覧
/// </summary>
TouchCollection touches;

The "TouchPanelCapabilities structure" is a structure that can have the functions of the touch panel itself as parameters. It is used to check whether the touch panel can be used during updating.

The TouchCollection structure contains a list of information about the currently touched state. If multiple touches are detected, multiple "TouchLocation structures" can be retrieved. More on that later.

Program - Loading Fonts

Before drawing text on the screen, add the font definition to your content project and load it with the Game.LoadContent method. This is not directly related to touch panels, so I will omit a detailed explanation.

Figure 2: Add "Font.spritefont" to your content project

// フォントをコンテンツパイプラインから読み込む
font = Content.Load<SpriteFont>("Font");

Program - Get touch panel information

Get the touch panel information inside the Game.Update method.

// タッチパネルの機能情報を取得
capabilities = TouchPanel.GetCapabilities();

// タッチパネルが使用可能であるかチェック
if (capabilities.IsConnected)
{
  // 現在のタッチパネルの入力情報を取得
  touches = TouchPanel.GetState();
}

You can call the "TouchPanel.GetCapabilities" method to get the function information of the touch panel. There are two types of information that can be acquired: "Can the touch panel be used?" and "Maximum number of touchpoints that can be acquired by the touch panel." Neither of them changes during the game, so I think there is no problem if you get it per the Game.Initialize method, but in the future, devices that can remove the touch panel (touch panel that can be connected to USB or actual devices other than Windows Phone) appear, and it is described in the Update method.

Also, there is no need to check whether the touch panel can be used because it can always be used on Windows Phone 7, but when sharing code with Windows and Xbox 360, these hardware are not required input interfaces and need to be checked.

If you can confirm that the touch panel is available in the "TouchPanelCapabilities.IsConnected" property, the "TouchPanel.GetState" method gets the current touch state.

Originally, some operation processing is performed after acquiring touch information, but in this sample, information is only displayed, so no further processing is done in the Game.Update method.

Program - Get the maximum number of touchpoints you can get

The maximum number of touchpoints that can be retrieved is roughly fixed, so it is not often necessary to retrieve them during the game, but you can get the maximum number of touchpoints that can be retrieved from the TouchPanelCapabilities.MaximumTouchCount property. For example, if the MaximumTouchCount property is "4", you cannot get the fifth position information by touching the touch panel with five fingers.

The number of acquisitions for the smartphone "HTC 7 Trophy" used in this sample was "4". (By the way, XNA Game Studio 4.0 is defined to always return 4.) The Windows Phone 7 specification says it is more than 4 points, so it does not return less than 4.)

// タッチ可能な最大数を表示
spriteBatch.DrawString(font,  
                       "MaximumTouchCount : " +
                         capabilities.MaximumTouchCount,
                       new Vector2(20, 50),
                       Color.LightGreen);

Figure 3: TouchPanelCapabilities.MaximumTouchCount Property Retrieval Count

Program - Get touch information

The retrieved TouchCollection structure contains multiple touch information. For example, if you touch with two fingers, it usually contains two touches.

The number of touch information can be retrieved in the TouchCollection.Count property. Touch information is repeatedly displayed in the for statement as many times as it is obtained. Originally, there is no problem to loop with foreach, but since we will use the array index (int index) in the later explanation, we loop it with for.

// タッチ情報の数だけループする
for (int index = 0; index < touches.Count; index++)
{
  // 指定したインデックスのタッチ情報取得
  TouchLocation tl = touches[index];

  // タッチ情報を可視化
  string mes = "Index : " + index + Environment.NewLine +
               "Id : " + tl.Id + Environment.NewLine +
               "Position : " + tl.Position + Environment.NewLine +
               "State : " + tl.State;

  // 文字の描画
  spriteBatch.DrawString(font,
                         mes,
                         new Vector2(30 + (index % 4) * 10,
                                     80 + index * 140),
                         Color.White);
}

Now, each touch information is obtained as a "TouchLocation" structure. If it is looping with for, it can be obtained with "TouchCollection[index]". (You can also get the "TouchLocation" structure directly in foreach.)

The following four types of information can be obtained from the "TouchLocation" structure, and each information is displayed in the sample.

When you actually run it, the touch information is displayed as shown below. Since you are touching with three fingers, you can see three touches. By the way, if you touch with five fingers, you'll only see up to four. (Because the maximum number of acquisitions is 4)

Figure 4: Touching with three fingers

Program - Things to watch out for when retrieving touch information 1 (index and ID)

I mentioned above that there are things to keep in mind when dealing with touch information, but the first is "index" and "ID". An index simply refers to an array index.

I think it is easier to understand if you try it on an actual machine than explain it in writing. For example, let's say you operate with two fingers, "index finger" and "middle finger", and touch with your index finger first.

Figure 5: Touch with index finger

As shown on the screen, the index will be "0". Then touch it with your middle finger.

Figure 6: Middle finger touch

In this state, you are touching with two fingers, so two pieces of information are displayed. The index touched with the middle finger is "1".

Let's lift our index finger here. Then the touch information will look like the following.

Figure 7: Release your index finger

As you may have noticed by now, the index of the information you are touching with your middle finger is "0". If you manage the character by index when you move it, the operation above will lead to the following actions.

・ Move the character "A" with your index finger
↓
・ Move the character "B" with the middle finger in that state.
↓
・ Release the index finger while moving the character with the middle finger
↓
・ The character controlled by the middle finger suddenly switches to "A"

Another phenomenon is that if you repeatedly release or touch one of the two fingers while touching them, the drawing text of the first touch information may flicker. This is because the second finger interrupts the index "0" the moment you touch or release it.

As mentioned above, the index and the order of touch information when multi-touch do not match the order in which they are touched. Therefore, if you want to perform any operations, do not manage them in the TouchCollection index.

What should be managed is that the "TouchLocation" structure has an "Id" property, and the Id is rewritten with a new value every time it is touched, but the relationship between the touchpoint and the ID is guaranteed while touching, so we will manage it.

Of course, this does not mean that you have to manage it by ID, and in some cases, the touch position alone is sufficient. For example, if you just touch a button on the screen, you can determine that you pressed a button by checking the touch position regardless of ID. However, when including drag processing, the position always changes, so I think it should still be determined by ID.

Program - Things to keep in mind when acquiring touch information 2 (number of touch information acquired)

Although it is not included in the program of the article, the sample draws the string "GetMaxTouchCount" at the bottom of the screen and "the maximum number of touch information acquired at the same time". This has displayed the maximum number of touch information acquired by the "TouchPanel.GetState" method in the past, but in this test machine, the "TouchPanelCapabilities.MaximumTouchCount" property is 4, so the maximum value displayed here should usually be 4. If you put five fingers on it, it will still be 4.

Figure 8: Five-finger number

Let's do a little experiment. Try repeating the touch panel with multiple fingers at high speed. It depends on the test machine, but the number may exceed 4 before you know it.

Figure 9: GetMaxTouchCount is greater than 4

In fact, the touch information acquired by the "TouchPanel.GetState" method is not the touch information at the moment of calling the "TouchPanel.GetState" method, but the moment you touch it and the moment you release it from the last update timing are cached. Therefore, even though up to four touches can be detected at the same time, touching the same finger again is treated as a separate touch, so more touch information may be obtained than TouchPanelCapabilities.MaximumTouchCount.

Therefore, even if TouchPanelCapabilities.MaximumTouchCount is 4, if the number of array elements of touch-related data is fixed by 4, an index overage error may occur depending on the processing method.

By the way, in the help of XNA Game Studio, the maximum value of the TouchCollection.Count property is described as "8", so if you want to fix the number of elements in the array, you should set the number of elements to at least 8. (If you think about future version upgrades, you may want to include a guard clause so that the index does not go over.)

Summary of this sample

This time, we examined various parameters for the information obtained from multi-touch. We also confirmed that there are some points to keep in mind when acquiring touch panel information. Based on this knowledge, I would like to explain a sample that actually uses multi-touch.