VRHandsFrame

日本語ページ

Online Manual v1.3.0 (Update at 2022/06/16)

Index

  1. Application Startup
  2. Basic Operation
  3. Change Settings on SteamVR Dashboard
  4. Basic Settings
  5. Image Saving Function / Settings
  6. Translation and Text Recognition Function / Settings
  7. 2D Code Reading Function / Settings
  8. Advanced Settings
  9. Transfer of configuration data

Appendix

1. Application Startup

After extracting the compressed file downloaded from booth, execute “VRHandsFrame.exe” in the folder.
After execution, SteamVR will launch at the same time.
(If SteamVR is already running, only the application will launch)

* the application does not work properly if it is placed in a location containing multi-bytes characters.
please be careful where you install the application.

The first time VRHandsFrame is launched, it will be registered in the startup settings of SteamVR.
After the second time, it will automatically launch linked to the startup of SteamVR.

Each direct execution of VRHandsFrame.exe will overwrite the startup settings.
If you have moved the location of the app folder or want to run a different version of the app, run VRHandsFrame.exe directly from the target folder to override the startup settings.

If you wish to remove the auto startup settings, please configure the SteamVR settings.

2. Basic Operation

[Step 1]
Hold both hands close together while pressing the grip buttons on the two-handed controller.
Hold a hand-sign pose as if you were forming a frame.

画像に alt 属性が指定されていません。ファイル名: baisc_step1.png

[Step 2]
While you keep the framing pose, a white frame continue to generate in a clockwise direction.
When frame generation is complete, the frame color changes.

画像に alt 属性が指定されていません。ファイル名: basic_step2.gif

[Step 3]
After the frame color changes, the assigned function becomes available.
When the controller is moved with the grip held down, the frame scales accordingly.

画像に alt 属性が指定されていません。ファイル名: basic_step3.gif

Once the frame is fully generated, the angle and positioning of the controller can be changed.

[Step 4]
Move the controller to bring the object into the frame.
Pressing the trigger button on either the left or right controller while holding the grip down executes the assigned function.

画像に alt 属性が指定されていません。ファイル名: basic_step4.gif

If both left and right controller triggers are pressed, the frame is fixed at the position at the moment of pressing, and the timer function is activated. (Default timer is 5 seconds)

Two different functions can be performed depending on the position of the two hands at the start of frame generation.
(Image saving and translation functions are assigned by default)

画像に alt 属性が指定されていません。ファイル名: basic_step4_2.gif

If the trigger/grip presses interfere with actions of each VR game, you may avoid that interference by changing the “Reverse Trigger Roles and Grip Roles” or “Use Button/Trigger Touch Action” items in the Advanced Settings.

Caution
If the area in the frame is extremely too narrow or too wide, the area cannot be determined.
The frame color is used to determine the area. If the RGB values of the background/object are extremely similar to the frame color, it may fail to recognize the area.

3. Change Settings on SteamVR Dashboard

Selecting the VRHandsFrame icon on the SteamVR Dashboard with the pointer will bring up the settings screen. Various settings can be changed here.

To change the settings, select the red button “保存 Save” on the settings screen.

VRHandsFrame settings can be switched between “Global Settings” and “Application Settings”.

[Global Settings]:
This setting is commonly available regardless of the game being launched.

[Application Settings]:
Individual settings that can be saved for each game/application running on SteamVR.

To switch settings, select the green button on the dashboard “Use Global settings” or “Use Application Settings”.

*Caution
If you switch settings before selecting the “保存 Save” button, the settings before saving will be lost. Please be careful.

4. Basic Settings

Select the “基本設定” tab on the dashboard to change basic settings.

[Mode]
Select the function to be assigned to the hand sign shown on the left from “Photo”, “Translation”, and “2D Code”.

[Frame Color]
Sets the color of the frame to be assigned to the hand sign shown on the left.

[Hand-Sign Duration]
Sets the duration required to maintain the hand sign before the function becomes available.

[Frame Disable Duration]
Sets the duration from when the hand sign is released until the frame is completely erased (returns to the standby state).

[Timer Seconds]
Sets the number of seconds to count when using the timer function.

[Sound Effect]:
Sets sound effect ON/OFF.

[Referenced Eye]:
Choose which eye’s view to refer.

5. Image Saving Function / Settings

The area within the frame is cropped as an image.
Since the VR view is cropped as it is, the UI of another overlay application may be reflected in the image.

In the default state, the images are saved to the folder “VRHandsFrame” in My Pictures. In addition, captured images are saved in subfolders for each application.

Please Select to the “Photo” tab to change settings.

[Image Save Path]:
You can set the path to save the image. Copy the path you want to save on the desktop and select the “Paste” button. If you want to restore the default settings, select the “Default” button.

6. Translation and Text Recognition Function / Settings

Recognizes characters reflected in the image within the frame and translates them into the respective languages.

After the translation results are displayed, you can page forward with the right hand trigger and backward with the left hand trigger. To exit the translation function, press the right trigger at the end of the page, or press both hand triggers simultaneously in the middle of the page.

The default character recognition engine uses the engine provided by the Windows system.
Therefore, if you want to use the default character recognition, you need to install Windows language packs for each language. Please refer to the following manual to install the language pack for the character recognition target.

Installing Language Packs for Windows

VRHandsFrame’s translation functionality allows users to share a single translation server by default. The translation server uses a service called Google App Script (GAS), which limits the number of translations (5000 times/day). This means that, by default, 5000 times/day of translation resources are shared by all VRHandsFrame users.

You can individually set up your own personal GAS translation server or use some translation engines (DeepL, Azure) contracted by yourself.

You can also setup personal translation servers.
By setting up a translation server as GAS settings, each user can use the full 5000 times/day translation resource.

For more information on setting up a personal translation server, please refer to the following additional manuals

Appendix:Deployment of personal translation servers



In Addition, VRHandsFrame also enables highly accurate image and text recognition using the Google Cloud Vision API. The installation of a language pack is not required to use the Vision API, but additional configuration is necessary.

Due to the use of Google’s service, there is a limit of 1000 times/month of character recognition that can be used for free, and 1001 times or more will be charged to Google’s service.

See additional manuals below for details.

Appendix:Use Google Cloud Vision API

Character Recognition and Translation Engine Comparison Table

Please Select the “Translation” tab to change the settings on the dashboard.

[Language from Translate]:
Set the translation source language. (24 languages)

[Language to Translate]:
Set the translation target language. (24 languages)

When the source and target languages are the same, the text recognition results are displayed. Please use this function to check the recognition accuracy.

[Font Size]:
Select the font size for the translation results (S / M / L)

[Translation Engine]
Set the translation engine. By default, Google App Script (GAS) is used.
If you wish to use an additional translation engine, set the API key you obtained in the following Config file.

[Translation Engine Settings File]
\VRHandsFrame_Data\StreamingAssets\config\translation_api.json

<Default>:
Use Google App Script(GAS). Default setting.

<DeepL>:
Use DeepL API (Free/Pro). API key setting is required.

<Azure>:
Use Microsoft Azure Translator API. API key setting is required.
Select “global” for the resource region in Azure Settings Web Page.

<Mix>:
Basically, use DeepL API.
For languages not supported by DeepL API (Traditional Chinese, Korean), use Azure API or GAS.

[GAS Key]:
Set up a Google Apps Script API Key (GAS Key) for your personal translation server.
Please refer to the additional manual for setting up.



7. 2D Code Reading Function / Settings

The 2D code displayed on the image in the frame is read.

Place the 2D code in the frame from the front, avoiding an oblique angle as much as possible. If the reading result is a URL, press the trigger to open the browser on the desktop.

Please go to the “2D Code” tab to change the settings on the dashboard.

[Font Size]:
Select font size for 2D code reading results (S / M / L)



8. Advanced Settings

You can change the advanced settings of VRHandsFrame.

To change the settings, select the “詳細設定” tab.
Since there are multiple setting screens, each selection switches the page.

Advanced Settings 1

[Reverse Trigger Roles and Grip Roles]:
Turning the setting ON will swap the roles of the trigger and grip on the VRHandsFrame.

[Use Button/Trigger Touch Action]:
When the setting is turned on, the touch of the “button” and “trigger” can be assigned to perform hand sign conditions and functions instead of pressing the “grip” and “trigger”.
* Some controllers do not support touch operation of triggers due to hardware specifications.

Binding of various controllers when [Use Button/Trigger Touch Action] is turned on

ControllersGenerate the frameFix the frame
Oculus Touch ControllerTouch right A + left X buttonTouch trigger
Vive ControllerTouch Right + left trackpad
Touch trigger
Index ControllerTouch right A + left A button
Touch trigger
Vive Cosmos ControllerTouch right A + left X button
Press trigger
Windows MR ControllerTouch Right + left trackpad
Press trigger

[Disable Functions while Running Current App]:
Turning the setting ON will disable VRHandsFrame all functionality when using the current app.
(assumes use of Application Settings)

[Adjust Scale of Generating Frame]:
Adjusts the size of the generated frame.

The larger this value is, the closer the frame generation criteria will be to the midpoint of both hands.
The smaller this value is, the farther the frame generation criteria will be to the midpoint of both hands.
(This is intended to be used as a workaround for cases where the frame is covered by the avatar’s hands.)

[Threshold of Detecting Frame]:
Sets the threshold for frame detection. (Default value: 5)
If frames cannot be detected due to differences in CPU or HMD rendering processes, increasing the threshold may improve the problem.

[Ignore Brightness Settings in OVR Advanced Settings]:
Ignore the brightness settings of another overlay application “OVR Advanced Settings” while VRHandsFrame functions are running.



Advanced Settings 2

[Use Google Cloud Vision API for Text Recognition]:
After turning on the setting and performing additional configuration, Google Cloud Vision API can be used for text recognition.
Please refer to the additional manual for how to configure Vision API.

[Estimate language When Using Vision API [beta]]
Estimates the language of text recognition when using Vision API. Note that the language may be recognized as different from the expected language.
When enabled, the [Language from translation source] setting is ignored.
Languages other than supported languages can also be recognized and translated, but the incidence of misrecognition will be higher.

9. Transfer of configuration data

When upgrading VRHandsFrame or when you want to install VRHandsFrame on a different PC, you can take over the configuration data.

Each VRHandsFrame configuration data is saved in json format files in the following folder.
If you want to take over the configuration data, copy the entire [folder for configuration] to the same location in the new VRHandsFrame, or copy the json files individually.

[folder for configuration]
\VRHandsFrame_Data\StreamingAssets\config

[Global Settings File]
\VRHandsFrame_Data\StreamingAssets\config\global.json

[Application Settings File]
\VRHandsFrame_Data\StreamingAssets\config\[Application Name]\settings.json

[Vision API Settings File]
\VRHandsFrame_Data\StreamingAssets\config\vision_api.json

[Translation Engine Settings File]
\VRHandsFrame_Data\StreamingAssets\config\translation_api.json