Gtuner IV 1.07 PUBLIC RELEASE: Computer Vision
Gtuner IV 1.07 PUBLIC RELEASE: Computer Vision
-- UPDATE (March 14th, 2020) --
Gtuner IV 1.07 Public Release
-- UPDATE (March 6th, 2020) --
Gtuner IV release candidate is now available for testing. In this 1.07 final testing release is now possible add an URL/link for donation when publishing or updating a work (thank you Mad for the idea). With that the author can receive donations from users as gratitude for the hard work and time spend. The donation button will be shown below the work information on the right side and will work on all Gtuner versions (current and previous), as well as at the ConsoleTuner.com Online Resource viewer.
Gtuner IV 1.07RC Changelog:
* Qt framework updated to 5.14.1
* Preferences: Added option to manually enable automatic scaling for high DPI monitors
* Session: The order of files opened in the GPC Script IDE are now correctly restored
* Memory Slot: Fixed issue with the configuration button not being displayed after script drop
* Gtuner IV: Added splash screen for early indication Gtuner IV is launching
* Computer Vision panel for view video capture, select source, run CV scripts, save frame as image
* Computer Vision: Input source configuration interface (device, stream, display, window, null)
* Computer Vision: integration with Python executable library and OpenCV-Python module
* Preferences: Added CV pane to select Python location and check if files needed were found
* Preferences (Test and Debug): Added option to automatic open the CV panel for GCV scripts
* File Explorer: Added icon, enabled listing and opening of GCV script files (*.py)
* Gtuner Menu and Toolbar: added functionalities to create and manage GCV script files (*.py)
* Code Editor: Added custom lexer, highlight, auto-completion for GCV script files (*.py)
* Computer Vision: Accept GPC script commands "GCMD:ComputerVision.[Stop|Start|Start:<gcvfile>]"
* GPC Compiler: New functions gcv_ready() and gcv_read() to receive feedback from GCV scripts
* Publishing: Accept GCV scripts and referred supporting files for CV algorithms (images, xml)
* Online Resources: Changed category 'Macro' to 'Misc' for list GMK macros and GCV scripts
* Gtuner Update: Launch Gupdate helper application detached from Gtuner IV process
* Computer Vision: Built-in gtuner Python module to read the input, output, LED and FFB states
* Computer Vision: Use OpenGL for the frame viewport (faster frame drawing, less CPU usage)
* Computer Vision: Fixed horizontal shifting in the export frame to image operation {GitHub#359}
* Header <titanone.gph>: Fixed error in the algorithm of stickize() function {GitHub#360}
* Output Panel: Print debug messages related with Bluetooth Module connection errors
* Code Editor: Auto select corresponding file type for 'Save As' and 'Save Copy As' {GitHub#358}
* Gtuner Mac: Workaround for OSX Catalina not reporting removal of device, causing Gtuner to crash
* Gtuner Mac: Computer Vision, with default kern.sysv.shmmax viewport is 720p, increase for 1080p
* Gtuner Mac: Several GUI layout fixes and application style changed to 'Fusion' for better look
* Publishing: Added extra field for the author provide URL for donations (optional) {GitHub#364}
* Code Editor: Fixed bug with 'Save As' and 'Save Copy As' for GCV script files {GitHub#363}
* GPC Compiler: Added MOUSE_ACPAN definition to mouse.gph header file (horizontal scroll)
-- UPDATE (February 25th, 2020) --
Gtuner 1.07B-20 for Windows and now for Mac OS X, download links:
-- UPDATE (February 19th, 2020) --
New revision of the Gtuner IV 1.7-BETA is now available, download link [removed, official release out.]
Changes on this revision (1.7.1.15)
Implemented built-in 'gtuner' Python module to read from a GCV Script the input, output, LED and FFB states of the devices connected to the Titan Two.
GCV code example:
-- UPDATE (February 18th, 2020) --
New revision of the Gtuner IV 1.7-BETA is now available, download link: [removed, official release out.]
Changes on this revision (1.7.1.10)
Improved Computer Vision Toolbar:
Support for wide range of video inputs:
- Video Capture Devices (such as Capture Cards, Webcams)
- Network Video Streams (such as RTSP/H264 video stream)
- Display Capture (with support for multi-monitors)
- Window Capture (select the application to be captured)
- Null Video Input (run GCV scripts that does not requires video input)
Note: Display Capture and Window Capture requires PyWin32 module to be installed.
Fully customizable video input configuration:
For video capture and network streaming is now possible to manually choose the media backend. In the first beta release the backend in use was MS Media Foundation. If your capture card didn't work at that time, try again using MS DirectShow, which is an old backend, but has support for a greater variety of video capture devices.
Easy setup: Click on "Detect Devices, "Detect Displays" or "Detect Windows" button, select one option and hit OK to fill all relevant configurations automatically.
Advanced Configurations:
For video capture and network streaming the key-value pair can be any from the OpenCV API, as defined here:
https://docs.opencv.org/4.2.0/d4/d15/gr ... f7c4f5704d
and here:
https://docs.opencv.org/4.2.0/dc/dfc/gr ... thers.html
For Display Capture the valid keys are:
enjoy!
-- ORIGINAL POST --
Gtuner IV 1.07 is now available for beta testing (Windows only).
This release adds a scriptable Computer Vision Interface to Gtuner IV, allowing the Titan Two users download, create and share computer vision algorithms (GCV scripts) that analyzes video streaming from games or others video sources such as webcams. The computer vision script can communicate with the Titan Two device to provide information for the running GPC Script.
The possibilities for this new feature are endless, to illustrate:
A example of GCV script is available on Gtuner's Online Resource, at "Misc Files" category.
GCV scripts are programmed in Python language and defined as a Python Class with name 'GCVWorker'. The class has a constructor '__init__' that is executed when the script loads, here is where the programmer can allocate, load and initialize any resource needed for the computer vision algorithm, as well as be informed of what will be the resolution of captured video frames. The destructor '__del__' is executed when the script is about to unload and should be used to free any allocated resource. The main method is named 'process' and is called whenever a new frame is captured. The process method can return up to two values: the frame to be displayed on Gtuner IV and/or an array of up to 255 bytes to be send to the Titan Two script. If the returning value for the frame is None, Gtuner IV will display the original captured frame. If the byte array is None, no data will be send to the Titan Two on that frame iteration.
Minimal GCV script example:
The Titan Two script can read the data from the computer vision algorithm by means of gcv_ready() and gcv_read(), as described here:
https://www.consoletuner.com/wiki/index ... :gcv_ready
https://www.consoletuner.com/wiki/index ... 2:gcv_read
Download the full beta release package from the following link: [removed, official release out.]
!!ATTENTION!!
Don't update your current Gtuner installation, otherwise you will get 'zlibwapi.dll was not found' error.
--
Any feedback on beta release will be very appreciated.
Thank you.
Gtuner IV 1.07 Public Release
-- UPDATE (March 6th, 2020) --
Gtuner IV release candidate is now available for testing. In this 1.07 final testing release is now possible add an URL/link for donation when publishing or updating a work (thank you Mad for the idea). With that the author can receive donations from users as gratitude for the hard work and time spend. The donation button will be shown below the work information on the right side and will work on all Gtuner versions (current and previous), as well as at the ConsoleTuner.com Online Resource viewer.
Gtuner IV 1.07RC Changelog:
* Qt framework updated to 5.14.1
* Preferences: Added option to manually enable automatic scaling for high DPI monitors
* Session: The order of files opened in the GPC Script IDE are now correctly restored
* Memory Slot: Fixed issue with the configuration button not being displayed after script drop
* Gtuner IV: Added splash screen for early indication Gtuner IV is launching
* Computer Vision panel for view video capture, select source, run CV scripts, save frame as image
* Computer Vision: Input source configuration interface (device, stream, display, window, null)
* Computer Vision: integration with Python executable library and OpenCV-Python module
* Preferences: Added CV pane to select Python location and check if files needed were found
* Preferences (Test and Debug): Added option to automatic open the CV panel for GCV scripts
* File Explorer: Added icon, enabled listing and opening of GCV script files (*.py)
* Gtuner Menu and Toolbar: added functionalities to create and manage GCV script files (*.py)
* Code Editor: Added custom lexer, highlight, auto-completion for GCV script files (*.py)
* Computer Vision: Accept GPC script commands "GCMD:ComputerVision.[Stop|Start|Start:<gcvfile>]"
* GPC Compiler: New functions gcv_ready() and gcv_read() to receive feedback from GCV scripts
* Publishing: Accept GCV scripts and referred supporting files for CV algorithms (images, xml)
* Online Resources: Changed category 'Macro' to 'Misc' for list GMK macros and GCV scripts
* Gtuner Update: Launch Gupdate helper application detached from Gtuner IV process
* Computer Vision: Built-in gtuner Python module to read the input, output, LED and FFB states
* Computer Vision: Use OpenGL for the frame viewport (faster frame drawing, less CPU usage)
* Computer Vision: Fixed horizontal shifting in the export frame to image operation {GitHub#359}
* Header <titanone.gph>: Fixed error in the algorithm of stickize() function {GitHub#360}
* Output Panel: Print debug messages related with Bluetooth Module connection errors
* Code Editor: Auto select corresponding file type for 'Save As' and 'Save Copy As' {GitHub#358}
* Gtuner Mac: Workaround for OSX Catalina not reporting removal of device, causing Gtuner to crash
* Gtuner Mac: Computer Vision, with default kern.sysv.shmmax viewport is 720p, increase for 1080p
* Gtuner Mac: Several GUI layout fixes and application style changed to 'Fusion' for better look
* Publishing: Added extra field for the author provide URL for donations (optional) {GitHub#364}
* Code Editor: Fixed bug with 'Save As' and 'Save Copy As' for GCV script files {GitHub#363}
* GPC Compiler: Added MOUSE_ACPAN definition to mouse.gph header file (horizontal scroll)
-- UPDATE (February 25th, 2020) --
Gtuner 1.07B-20 for Windows and now for Mac OS X, download links:
-- UPDATE (February 19th, 2020) --
New revision of the Gtuner IV 1.7-BETA is now available, download link [removed, official release out.]
Changes on this revision (1.7.1.15)
Implemented built-in 'gtuner' Python module to read from a GCV Script the input, output, LED and FFB states of the devices connected to the Titan Two.
GCV code example:
- Code: Select all
import cv2
import gtuner
class GCVWorker:
def __init__(self, width, height):
pass
def __del__(self):
pass
def process(self, frame):
# Read input states
stick_1_x = gtuner.get_actual(gtuner.STICK_1_X)
stick_1_y = gtuner.get_actual(gtuner.STICK_1_Y)
button_5 = gtuner.get_actual(gtuner.BUTTON_5)
button_8 = gtuner.get_actual(gtuner.BUTTON_8)
# Read output states
button_14 = gtuner.get_val(gtuner.BUTTON_14)
button_15 = gtuner.get_val(gtuner.BUTTON_15)
button_16 = gtuner.get_val(gtuner.BUTTON_16)
button_17 = gtuner.get_val(gtuner.BUTTON_17)
# Read LED and FFB states
led_1 = gtuner.led_get(gtuner.LED_1)
ffb_2 = gtuner.ffb_get(gtuner.FFB_2)
# Show state values
frame = cv2.putText(frame, '{0:+6.2f}'.format(stick_1_x), (60, 80), cv2.FONT_HERSHEY_SIMPLEX, 1, (255, 255, 255), 2, cv2.LINE_AA)
frame = cv2.putText(frame, '{0:+6.2f}'.format(stick_1_y), (60, 120), cv2.FONT_HERSHEY_SIMPLEX, 1, (255, 255, 255), 2, cv2.LINE_AA)
frame = cv2.putText(frame, '{0:+6.2f}'.format(button_5), (60, 160), cv2.FONT_HERSHEY_SIMPLEX, 1, (255, 255, 255), 2, cv2.LINE_AA)
frame = cv2.putText(frame, '{0:+6.2f}'.format(button_8), (60, 200), cv2.FONT_HERSHEY_SIMPLEX, 1, (255, 255, 255), 2, cv2.LINE_AA)
frame = cv2.putText(frame, '{0:+6.2f}'.format(button_14), (60, 240), cv2.FONT_HERSHEY_SIMPLEX, 1, (255, 255, 255), 2, cv2.LINE_AA)
frame = cv2.putText(frame, '{0:+6.2f}'.format(button_15), (60, 280), cv2.FONT_HERSHEY_SIMPLEX, 1, (255, 255, 255), 2, cv2.LINE_AA)
frame = cv2.putText(frame, '{0:+6.2f}'.format(button_16), (60, 320), cv2.FONT_HERSHEY_SIMPLEX, 1, (255, 255, 255), 2, cv2.LINE_AA)
frame = cv2.putText(frame, '{0:+6.2f}'.format(button_17), (60, 360), cv2.FONT_HERSHEY_SIMPLEX, 1, (255, 255, 255), 2, cv2.LINE_AA)
frame = cv2.putText(frame, '{0:+6.2f}'.format(led_1), (60, 400), cv2.FONT_HERSHEY_SIMPLEX, 1, (255, 255, 255), 2, cv2.LINE_AA)
frame = cv2.putText(frame, '{0:+6.2f}'.format(ffb_2), (60, 440), cv2.FONT_HERSHEY_SIMPLEX, 1, (255, 255, 255), 2, cv2.LINE_AA)
return (frame, None)
-- UPDATE (February 18th, 2020) --
New revision of the Gtuner IV 1.7-BETA is now available, download link: [removed, official release out.]
Changes on this revision (1.7.1.10)
Improved Computer Vision Toolbar:
Support for wide range of video inputs:
- Video Capture Devices (such as Capture Cards, Webcams)
- Network Video Streams (such as RTSP/H264 video stream)
- Display Capture (with support for multi-monitors)
- Window Capture (select the application to be captured)
- Null Video Input (run GCV scripts that does not requires video input)
Note: Display Capture and Window Capture requires PyWin32 module to be installed.
Fully customizable video input configuration:
For video capture and network streaming is now possible to manually choose the media backend. In the first beta release the backend in use was MS Media Foundation. If your capture card didn't work at that time, try again using MS DirectShow, which is an old backend, but has support for a greater variety of video capture devices.
Easy setup: Click on "Detect Devices, "Detect Displays" or "Detect Windows" button, select one option and hit OK to fill all relevant configurations automatically.
Advanced Configurations:
For video capture and network streaming the key-value pair can be any from the OpenCV API, as defined here:
https://docs.opencv.org/4.2.0/d4/d15/gr ... f7c4f5704d
and here:
https://docs.opencv.org/4.2.0/dc/dfc/gr ... thers.html
For Display Capture the valid keys are:
- Code: Select all
CAP_CROP_TOP
CAP_CROP_BOTTOM
CAP_CROP_LEFT
CAP_CROP_RIGHT
- Code: Select all
CAP_OFFSET_X
CAP_OFFSET_Y
enjoy!
-- ORIGINAL POST --
Gtuner IV 1.07 is now available for beta testing (Windows only).
This release adds a scriptable Computer Vision Interface to Gtuner IV, allowing the Titan Two users download, create and share computer vision algorithms (GCV scripts) that analyzes video streaming from games or others video sources such as webcams. The computer vision script can communicate with the Titan Two device to provide information for the running GPC Script.
The possibilities for this new feature are endless, to illustrate:
- - Automated detection of the game aiming mechanics for creation of optimized mouse to stick translators (yx ratio, dd factor, conversion curve, deadzone, etc).
- - - Detect and inform to the Titan Two script the equipped gear and game status (weapon model, weapon attachments, fire mode, shots fired, etc).
- - - Analyzing of the game minimap to alert the player about any important event (so the player can fully concentrate on the actual game and stop look to the minimap every couple seconds).
- - - Detect and register weapon recoil/spray pattern, so the collected data can be used for creation of anti-recoil scripts.
- - - Pattern detection and triggering of automated actions, such as:
- * auto switch to secondary weapon if detect the primary weapon was fully unloaded;
- - * auto block if an incoming attack is detected;
- - * auto fire if detected an enemy is aligned with the aiming reticle.
- - * and so on...
- * auto switch to secondary weapon if detect the primary weapon was fully unloaded;
A example of GCV script is available on Gtuner's Online Resource, at "Misc Files" category.
GCV scripts are programmed in Python language and defined as a Python Class with name 'GCVWorker'. The class has a constructor '__init__' that is executed when the script loads, here is where the programmer can allocate, load and initialize any resource needed for the computer vision algorithm, as well as be informed of what will be the resolution of captured video frames. The destructor '__del__' is executed when the script is about to unload and should be used to free any allocated resource. The main method is named 'process' and is called whenever a new frame is captured. The process method can return up to two values: the frame to be displayed on Gtuner IV and/or an array of up to 255 bytes to be send to the Titan Two script. If the returning value for the frame is None, Gtuner IV will display the original captured frame. If the byte array is None, no data will be send to the Titan Two on that frame iteration.
Minimal GCV script example:
- Code: Select all
import os
import cv2
class GCVWorker:
def __init__(self, width, height):
os.chdir(os.path.dirname(__file__))
self.gcvdata = bytearray([0x00])
def __del__(self):
del self.gcvdata
def process(self, frame):
return (frame, None)
The Titan Two script can read the data from the computer vision algorithm by means of gcv_ready() and gcv_read(), as described here:
https://www.consoletuner.com/wiki/index ... :gcv_ready
https://www.consoletuner.com/wiki/index ... 2:gcv_read
Download the full beta release package from the following link: [removed, official release out.]
!!ATTENTION!!
Don't update your current Gtuner installation, otherwise you will get 'zlibwapi.dll was not found' error.
--
Any feedback on beta release will be very appreciated.
Thank you.
ORIGINAL POST wrote:Hi All, This post is to give you guys an update on what I am working these past weeks.
First, a background to set the context. By design, one of primaries requirements for the Titan Two is the "native" support for keyboard and mouse gameplay, meaning the mouse and keyboard could be directly plugged to the device without rely on computer connection. This feature is long supported by the Titan Two.
However, find the best mouse conversion parameters is not an easy task and normally it is a very time consuming process when doing manually. Since early stages of the Titan Two development the plan is implement some sort of computer vision tool to automate this process and find the most optimal mouse settings for each game.
November last year I finally started work on this tool, as the development progressed I thought: why not have this tool integrated on Gtuner VI, so all Titan Two users could make use of it to find the best mouse settings? sounds good, lets do it.
Few weeks later, I had another thought: this tool can also be used for others purposes, such as analyze the game frames to provide valuable information for the Titan Two script (e.g. equipped weapon, fire mode, etc), or even more advanced usages like have as video input source a webcam and use computer vision algorithms to aid disabled person to play their favorite games (e.g. activate the jump action with movement of a hand).
The ideas didn't stop come, the possibilities are endless. However, having the computer vision algorithms hard coded on Gtuner IV would impose an insurmountable barrier for these possibilities, as it would direct depend on me to implement each.
Okay, let have the computer vision algorithms as Gtuner IV plugins, so anyone interested could make their own. Sounds good, but it would require installation of compilers, computer vision libraries and advanced knowledge of computer programming. Not an accessible approach.
So what about have scripts to implement the computer vision algorithms? This would definitively be a more accessible option. Moreover if fully integrated on Gtuner IV.
So I did and here is the result:
The computer vision scripts are based on the OpenCV library and programmed utilizing Python script language.
It is required to have Python 3 x64 installed, as well as the module 'OpenCV-Python':
At some point next week Gtuner VI 1.07 will be made available as beta release (in the beta stage will be Windows only).
ConsoleTuner Support Team
-
J2Kbr - General of the Army
- Posts: 20323
- Joined: Tue Mar 18, 2014 1:39 pm
Re: Computer Vision with Gtuner IV and Titan Two
Wow. That is a lot of work and great ideas. This is awesome.
-
UK_Wildcats - Brigadier General
- Posts: 2243
- Joined: Thu Jan 08, 2015 6:53 pm
Re: Computer Vision with Gtuner IV and Titan Two
J2Kbr wrote:However, find the best mouse conversion parameters is not an easy task and normally it is a very time consuming process when doing manually. Since early stages of the Titan Two development the plan is implement some sort of computer vision tool to automate this process and find the most optimal mouse settings for each game.
November last year I finally started work on this tool, as the development progressed I thought: why not have this tool integrated on Gtuner VI, so all Titan Two users could make use of it to find the best mouse settings? sounds good, lets do it.
This is great news, J2KBr!
I hope it does not require extensive coding knowledge to be able to work on mouse conversion parameters.
Awesome work, needless to say, on all the other features whereby experienced / advanced users will benefit greatly!
- My recommendation for learning C programming: 'C Programming Absolute Beginner's Guide'
- Create your own scripts with GPC2 Scripting
- Create your own scripts with GPC2 Scripting
-
jelly44 - Sergeant Major of the Army
- Posts: 220
- Joined: Tue Feb 05, 2019 3:49 pm
Re: Computer Vision with Gtuner IV and Titan Two
Wow, that is impressive! Great work, the possibilities are endless it's crazy!
Console tuner since my 1st controller.
Scripting, a game in the game.
Believe or dare, It's Titanic!
Scripting, a game in the game.
Believe or dare, It's Titanic!
-
Prototype - Major General
- Posts: 3251
- Joined: Sun Dec 16, 2012 1:43 pm
Re: Computer Vision with Gtuner IV and Titan Two
It's great to see this coming together J2K
2020 is shaping to be a big year for Titan Two.
2020 is shaping to be a big year for Titan Two.
-
antithesis - Colonel
- Posts: 1912
- Joined: Sat May 28, 2016 10:45 pm
Re: Computer Vision with Gtuner IV and Titan Two
I'm excited about this new tech. Keep us updated!
-
dognose - Master Sergeant
- Posts: 30
- Joined: Sun Nov 11, 2018 8:12 am
- Location: South Los Angeles, CA
Re: Computer Vision with Gtuner IV and Titan Two
Cant wait for this !
-
hammer - Sergeant Major
- Posts: 104
- Joined: Sun Sep 30, 2018 1:36 pm
- Location: Germany
Re: Gtuner IV 1.07 BETA (Scriptable Computer Vision Interfac
The beta release is now available, more details on the first post:
viewtopic.php?f=2&t=14717&p=95183#p95183
viewtopic.php?f=2&t=14717&p=95183#p95183
ConsoleTuner Support Team
-
J2Kbr - General of the Army
- Posts: 20323
- Joined: Tue Mar 18, 2014 1:39 pm
Re: Gtuner IV 1.07 BETA (Scriptable Computer Vision Interfac
J2Kbr wrote:The beta release is now available
Got this error after updating Gtuner IV:
ConsoleTuner Support Team || ConsoleTuner Discord || InputSense Discord
- Mad
- Major General
- Posts: 4536
- Joined: Wed May 22, 2019 5:39 am
Return to News and Announcements
Who is online
Users browsing this forum: No registered users and 87 guests