Select a model from its position on the screen

summary

It allows you to select the model at the position of the mouse cursor. When you hover the cursor over the model, the Hit text changes to True.

Operating environment

Prerequisites

Operating environment

How to work with the sample

substance

Convert screen coordinates to 3D spatial coordinates

You may want to select a model in 3D space with a mouse. In this case, it is necessary to convert the two-dimensional coordinate point on the screen to the three-dimensional coordinate where the model exists and perform a hit judgment.

However, in order to extend the element from 2D to 3D, it is not possible to find a point with 3D coordinates from 2D screen coordinates of only X and Y. For example, if you imagine actually clicking on the screen, you will understand that it is not possible to determine whether the position in 3D space when you click is in front of the object, the object itself, or behind the object.

Therefore, instead of representing the clicked position as a dot, it is treated as a line stretched from the camera position in the direction of the click. By performing collision detection between the line and the object, it is possible to select a model. By the way, line parameters can be handled in XNA by a structure called Ray.

Get the position in 3D space from the position of the screen

XNA does not have a method to find a line in the direction of a click on the screen. However, since it is possible to find a point in 3D space by specifying the screen coordinates and depth, it is possible to find a line by connecting the position of the camera and the 3D space coordinate point transformed at a specific depth.

Finding object-space coordinates from screen-space coordinates is easy to do using the "Viewport.Unproject" method.

// ビューポートを取得
Viewport viewport = this.GraphicsDevice.Viewport;

// スクリーンの位置を Vector3 で作成 (Z は 1.0 以下を設定)
Vector3 screenPosition = new Vector3(this.markPosition, 1.0f);

// スクリーン座標を３次元座標に変換
Vector3 worldPoint = viewport.Unproject(screenPosition,
                                        this.projection,
                                        this.view,
                                        Matrix.Identity);

The first argument is a Vector3 with screen coordinates and depth. Set X, Y to the coordinates of the screen, and Z to the depth value. The depth depends on the "nearPlaneDistance" and "farPlaneDistance" parameters of the projection matrix, where you can specify 0.0f to find the distance from the camera position to nearPlaneDistance, and 1.0f to determine the distance from the camera position to farPlaneDistance.

The second argument is the projection matrix, and the third argument is the view matrix.

You can find an object-space vector as the return value.

Viewport.Unproject method

Projects a vector from screen space into object space.

Create a Ray

Line parameters can be Ray structures. The first argument to the constructor is the starting point of the ray, and the second argument is the orientation of the ray.

Set the position of the camera as the starting point and calculate the orientation by subtracting the position of the camera from the 3D space coordinates that have already been converted to orientation. The orientation is set to a unit vector using the Vector3.Normalize method.

// マークが指す方向へのレイを作成
Ray ray = new Ray(this.cameraPosition,
                    Vector3.Normalize(worldPoint - this.cameraPosition));

Ray constructor

Create an instance of the "Ray" structure that contains the parameters of the line.

Ball and ray hitbox

The ModelMesh class loaded from the content pipeline contains sphere data that encompasses the mesh, called the BoundingSphere property. By specifying the Ray you just created in the Intersects method of this class, you can check if the sphere and ray are colliding.

In the event of a collision, the distance between the start of the ray and the collision point is returned. If there is no collision, null is returned, so the sample checks whether there is a collision by null judgment.

However, this method assumes that the model is located at the origin. If you are moving the model, you will need to transform the rays as the model moves.

By the way, this sample model is a sphere, so I think it can be accurately determined.

// 球とレイとの当たり判定を行う
this.isHit = false;
foreach (ModelMesh mesh in this.model.Meshes)
{
    if (mesh.BoundingSphere.Intersects(ray) != null)
    {
        // 球とレイは交差している
        this.isHit = true;
        break;
    }
}

BoundingSphere.Intersects method

Collision detection between the inclusive ball and the ray is performed.

All Codes

using System;
using System.Collections.Generic;
using System.Linq;
using Microsoft.Xna.Framework;
using Microsoft.Xna.Framework.Audio;
using Microsoft.Xna.Framework.Content;
using Microsoft.Xna.Framework.GamerServices;
using Microsoft.Xna.Framework.Graphics;
using Microsoft.Xna.Framework.Input;
using Microsoft.Xna.Framework.Media;
#if WINDOWS_PHONE
using Microsoft.Xna.Framework.Input.Touch;
#endif

namespace ModelSelectByScreenPosition
{
    /// <summary>
    /// ゲームメインクラス
    /// </summary>
    public class GameMain : Microsoft.Xna.Framework.Game
    {
        /// <summary>
        /// グラフィックデバイス管理クラス
        /// </summary>
        private GraphicsDeviceManager graphics = null;

        /// <summary>
        /// スプライトのバッチ化クラス
        /// </summary>
        private SpriteBatch spriteBatch = null;

        /// <summary>
        /// スプライトでテキストを描画するためのフォント
        /// </summary>
        private SpriteFont font = null;

        /// <summary>
        /// モデル
        /// </summary>
        private Model model = null;

        /// <summary>
        /// マーク
        /// </summary>
        private Texture2D mark = null;

        /// <summary>
        /// マーク画像の中心位置
        /// </summary>
        private Vector2 markCenterPosition = Vector2.Zero;

        /// <summary>
        /// マークの位置
        /// </summary>
        private Vector2 markPosition = new Vector2(100.0f, 100.0f);

        /// <summary>
        /// モデルへの当たり判定フラグ
        /// </summary>
        private bool isHit = false;

        /// <summary>
        /// カメラの位置
        /// </summary>
        private Vector3 cameraPosition = new Vector3(0.0f, 0.0f, 10.0f);

        /// <summary>
        /// ビューマトリックス
        /// </summary>
        private Matrix view;

        /// <summary>
        /// プロジェクションマトリックス
        /// </summary>
        private Matrix projection;


        /// <summary>
        /// GameMain コンストラクタ
        /// </summary>
        public GameMain()
        {
            // グラフィックデバイス管理クラスの作成
            this.graphics = new GraphicsDeviceManager(this);

            // ゲームコンテンツのルートディレクトリを設定
            this.Content.RootDirectory = "Content";

#if WINDOWS_PHONE
            // Windows Phone のデフォルトのフレームレートは 30 FPS
            this.TargetElapsedTime = TimeSpan.FromTicks(333333);

            // バックバッファサイズの設定
            this.graphics.PreferredBackBufferWidth = 480;
            this.graphics.PreferredBackBufferHeight = 800;

            // フルスクリーン表示
            this.graphics.IsFullScreen = true;
#endif
        }

        /// <summary>
        /// ゲームが始まる前の初期化処理を行うメソッド
        /// グラフィック以外のデータの読み込み、コンポーネントの初期化を行う
        /// </summary>
        protected override void Initialize()
        {
            // ビューマトリックス
            this.view = Matrix.CreateLookAt(
                        this.cameraPosition,
                        Vector3.Zero,
                        Vector3.Up
                    );

            // プロジェクションマトリックス
            this.projection = Matrix.CreatePerspectiveFieldOfView(
                        MathHelper.ToRadians(45.0f),
                        (float)this.GraphicsDevice.Viewport.Width /
                            (float)this.GraphicsDevice.Viewport.Height,
                        1.0f,
                        100.0f
                    );

            // コンポーネントの初期化などを行います
            base.Initialize();
        }

        /// <summary>
        /// ゲームが始まるときに一回だけ呼ばれ
        /// すべてのゲームコンテンツを読み込みます
        /// </summary>
        protected override void LoadContent()
        {
            // テクスチャーを描画するためのスプライトバッチクラスを作成します
            this.spriteBatch = new SpriteBatch(this.GraphicsDevice);

            // フォントをコンテンツパイプラインから読み込む
            this.font = this.Content.Load<SpriteFont>("Font");

            // モデルを作成
            this.model = this.Content.Load<Model>("Model");

            // ライトとビュー、プロジェクションはあらかじめ設定しておく
            foreach (ModelMesh mesh in this.model.Meshes)
            {
                foreach (BasicEffect effect in mesh.Effects)
                {
                    // デフォルトのライト適用
                    effect.EnableDefaultLighting();

                    // ビューマトリックスをあらかじめ設定
                    effect.View = this.view;

                    // プロジェクションマトリックスをあらかじめ設定
                    effect.Projection = this.projection;
                }
            }

            // マーク作成
            this.mark = this.Content.Load<Texture2D>("Mark");

            // マークの中心位置
            this.markCenterPosition = new Vector2(this.mark.Width / 2, this.mark.Height / 2);
        }

        /// <summary>
        /// ゲームが終了するときに一回だけ呼ばれ
        /// すべてのゲームコンテンツをアンロードします
        /// </summary>
        protected override void UnloadContent()
        {
            // TODO: ContentManager で管理されていないコンテンツを
            //       ここでアンロードしてください
        }

        /// <summary>
        /// 描画以外のデータ更新等の処理を行うメソッド
        /// 主に入力処理、衝突判定などの物理計算、オーディオの再生など
        /// </summary>
        /// <param name="gameTime">このメソッドが呼ばれたときのゲーム時間</param>
        protected override void Update(GameTime gameTime)
        {
            // キーボードの情報取得
            KeyboardState keyboardState = Keyboard.GetState();

            // ゲームパッドの情報取得
            GamePadState gamePadState = GamePad.GetState(PlayerIndex.One);

            // Xbox 360 コントローラ、Windows Phone の BACK ボタンを押したときに
            // ゲームを終了させます
            if (gamePadState.Buttons.Back == ButtonState.Pressed)
            {
                this.Exit();
            }

            // 移動スピード
            float speed = 200.0f;

            // キーボードによるマークの移動
            if (keyboardState.IsKeyDown(Keys.Left))
            {
                this.markPosition.X -= speed * (float)gameTime.ElapsedGameTime.TotalSeconds;
            }
            if (keyboardState.IsKeyDown(Keys.Right))
            {
                this.markPosition.X += speed * (float)gameTime.ElapsedGameTime.TotalSeconds;
            }
            if (keyboardState.IsKeyDown(Keys.Up))
            {
                this.markPosition.Y -= speed * (float)gameTime.ElapsedGameTime.TotalSeconds;
            }
            if (keyboardState.IsKeyDown(Keys.Down))
            {
                this.markPosition.Y += speed * (float)gameTime.ElapsedGameTime.TotalSeconds;
            }

            // ゲームパッドによるマークの移動
            if (gamePadState.IsConnected)
            {
                this.markPosition.X += gamePadState.ThumbSticks.Left.X * speed *
                                       (float)gameTime.ElapsedGameTime.TotalSeconds;
                this.markPosition.Y -= gamePadState.ThumbSticks.Left.Y * speed *
                                       (float)gameTime.ElapsedGameTime.TotalSeconds;
            }

            // マウス処理
            MouseState mouseState = Mouse.GetState();

            if (mouseState.X >= 0 && mouseState.X < this.Window.ClientBounds.Width &&
                mouseState.Y >= 0 && mouseState.Y < this.Window.ClientBounds.Height &&
                mouseState.LeftButton == ButtonState.Pressed)
            {
                // マウスがウインドウ内にあればマウスの位置を優先する
                this.markPosition = new Vector2(mouseState.X, mouseState.Y);
            }

            // ビューポートを取得
            Viewport viewport = this.GraphicsDevice.Viewport;

            // スクリーンの位置を Vector3 で作成 (Z は 1.0 以下を設定)
            Vector3 screenPosition = new Vector3(this.markPosition, 1.0f);

            // スクリーン座標を３次元座標に変換
            Vector3 worldPoint = viewport.Unproject(screenPosition,
                                                    this.projection,
                                                    this.view,
                                                    Matrix.Identity);

            // マークが指す方向へのレイを作成
            Ray ray = new Ray(this.cameraPosition,
                              Vector3.Normalize(worldPoint - this.cameraPosition));

            // 球とレイとの当たり判定を行う
            this.isHit = false;
            foreach (ModelMesh mesh in this.model.Meshes)
            {
                if (mesh.BoundingSphere.Intersects(ray) != null)
                {
                    // 球とレイは交差している
                    this.isHit = true;
                    break;
                }
            }

            // 登録された GameComponent を更新する
            base.Update(gameTime);
        }

        /// <summary>
        /// 描画処理を行うメソッド
        /// </summary>
        /// <param name="gameTime">このメソッドが呼ばれたときのゲーム時間</param>
        protected override void Draw(GameTime gameTime)
        {
            // 画面を指定した色でクリアします
            this.GraphicsDevice.Clear(Color.CornflowerBlue);

            // Zバッファを有効にする
            this.GraphicsDevice.DepthStencilState = DepthStencilState.Default;

            // モデルを描画
            foreach (ModelMesh mesh in this.model.Meshes)
            {
                mesh.Draw();
            }

            // スプライトの描画準備
            this.spriteBatch.Begin();

            // マーク描画
            this.spriteBatch.Draw(this.mark, this.markPosition,
                null, Color.White, 0.0f,
                this.markCenterPosition, 1.0f, SpriteEffects.None, 0.0f);

            // テキスト描画
            this.spriteBatch.DrawString(this.font,
                "Cursor Key Press or" + Environment.NewLine +
                "   MouseLeftButton Drag" + Environment.NewLine +
                "Hit : " + this.isHit,
                new Vector2(50.0f, 50.0f), Color.White);

            // スプライトの一括描画
            this.spriteBatch.End();

            // 登録された DrawableGameComponent を描画する
            base.Draw(gameTime);
        }
    }
}