Swing Vision AI line calls and Analysis

D

Demented

Semi-Pro
I've done quite a bit of experimenting using Playsight/In.Out/High speed Go Pro to tabulate some of my statistics. What's the particular reason that your serve analysis can't go back and look at all frames and update the speed metrics? I understand that if you're skipping frames to save power, the accuracy of serve speed will be way off during live processing but why not make a second pass later on? The numbers I saw were clearly off. The serves were registering 61 mph and I'd guess just based on loose eyeball test that they were moving in the upper 80's. Or optionally, when a service motion/start is detected why not have the AI process the next 30-60 frames in sequence to find the ball strike frame and the ball impact frame on the court? My only guess was that maybe you're not recording in high FPS to save disc space/power? The frames don't actually exist?
 
swupnil

swupnil

Rookie
@Demented we are certainly experimenting with a lot of the things you’re suggesting here. It is a delicate balance between battery life and processing speed (we don’t have the luxury of a massive computer like PlaySight).

We record the video at 30fps currently but will update this to 60fps in the near future.

We’re also currently working on shipping a major update that will bring tremendous accuracy improvements in serve speed / placement. It should be out by end of the month!

Short answer is that all of the things you’ve mentioned are in development - just takes some time to experiment, integrate into the app, test extensively and ship out to production, but we will have massive improvements over the next 3 months! I see no reason why we can’t become as accurate as a radar gun.
 
D

Demented

Semi-Pro
The frames not existing makes sense. I thought you were recording at 60 fps(since all the apple devices can) and then only processing every other frame to save compute cycles. I completely understand how the speeds are off with only 30 frames. The strike frame and impact frame landing before or on or after the frame@30 fps can inject a lot of variation. Being off by 1 or 2 frames on each end can introduce a 20% error margin. I used to analyze serves by hand and upgrading cameras from 30 fps to 60 fps made it so much easier.
 
swupnil

swupnil

Rookie
@Demented very cool! We've found something similar in our testing as well. We'd love to be able to run the A.I. at 60fps but that's quite a struggle. Running on the phone will require a bit more creativity but I'm confident we will find a solution :)

And yea it turns out recording at 60 fps instead of 30, while also running our A.I., is enough to crush the battery on devices like iPhone XR/XS. We still need to make some optimizations to the A.I. so that it can run faster, at which point 60fps over the course of a 2-hour match should be no problem at all. Coming soon for sure :)
 
Last edited:
D

Demented

Semi-Pro
Just digging into your serve speed calculations a bit more. Are you reporting average ball speed(as measured by distance/time) or using the advance formula to figure out head velocity at impact?

import math # published numbers from a now dead link KNOWN_SERVE_SPEED= 120. KNOWN_SPEED_AT_BOUNCE= 87.0 DISTANCE_TRAVELED=60. # A constant for formula: K= math.log(120./87)/60 def init_speed(nf, d=60, fps=29.97): speed = ( math.exp(K*d) - 1) / (5280.*K * nf) * fps * 3600 return round(speed*100)/100 def avg_speed(nf, d=60, fps=29.97): speed = d / (nf / fps ) * (3600./5280.) return round(speed*100)/100

I book marked a serve speed calculator a few years ago that used known ball serve velocity from pro match radar as compared to final velocity at bounce and built this formula. I've since then verified it using Playsight.
 
time_fly

time_fly

Hall of Fame
swupnil said:
@Demented very cool! We've found something similar in our testing as well. We'd love to be able to run the A.I. at 60fps but that's quite a struggle. Running on the phone will require a bit more creativity but I'm confident we will find a solution :)

And yea it turns out recording at 60 fps instead of 30, while also running our A.I., is enough to crush the battery on devices like iPhone XR/XS. We still need to make some optimizations to the A.I. so that it can run faster, at which point 60fps over the course of a 2-hour match should be no problem at all. Coming soon for sure :)
Click to expand...

It seems to me there's really 2 use cases for this software. Case 1 is someone who is checking the phone regularly during play, looking for statistics and updates. This might happen frequently during a coached session or a formal practice. But I suspect most people will fall into "case 2": going back and looking at the data after finishing playing. In this case, real-time processing is much less important. People might be willing to wait some time for processing or even off-load to a higher-powered device / tablet / computer in order to get better accuracy. For case 2, I think the option of full-resolution and full frame-rate processing, even if you have to throttle back the processor to conserve battery, recommend connecting to external power, or offload some work, will make users happier than faster results.

I also like the suggestion above of dynamically switching to full frame-rate (re)processing around the estimated time of key events like racquet contact and ball bounces.
 
D

Deleted member 369227

Guest
@time_fly

Fully agree... I can imagine scenarios where real-time processing is extremely valuable (e.g. if you are a coach evaluating a player and/or introducing some fixes in the stroke mechanics) but personally, I prefer high-level accuracy over real-time processing, even if it takes some time to process the 60fps video on the server.
 
Last edited by a moderator:
T

tennis tom

Legend
If it can call lines, would have loved it for yesterday's rec match with the oldsters--more dubious calls due to poor eyesight then the Nile has cataracts. Having instant replays for dubious line calls would revolutionize rec play! These folks weren't cheaten' they just can't see--many calls missed by 6"--they need Dr. Blake eye therapy exercises.
 
Last edited:
derick232

derick232

Rookie
I've definitely noticed an increase in ball speed accuracy in my last few uses. Where previously it would say my serve average was in the mid 50's, now it will register in the 80's. I still think that's quite slow for my serves but it's nice to see the improvement. It's been several years since I've had my serve gunned (previously at 111 mph), I'd really love for this to be accurate so I can see my improvement.
 
swupnil

swupnil

Rookie
@time_fly @Dragan thanks for the suggestion! The main issue is that it can take several hours to back up your video to the cloud so that would be a pretty significant delay if we processed in the cloud. If we want to process on device, that would also be problematic as you won't be able to exit the app while analyzing the video.

One option though... what if you could AirDrop the video to your Mac and analyze there? We could get the accuracy up quite a bit then :)

@Demented thanks for the formula! We actually track the entire trajectory in 3D so we could determine the velocity (x, y and z components) along any part of the hit or bounce trajectory. We currently report the velocity as it leaves your racket.
 
D

Deleted member 369227

Guest
swupnil said:
@time_fly @Dragan thanks for the suggestion! The main issue is that it can take several hours to back up your video to the cloud so that would be a pretty significant delay if we processed in the cloud. If we want to process on device, that would also be problematic as you won't be able to exit the app while analyzing the video.

One option though... what if you could AirDrop the video to your Mac and analyze there? We could get the accuracy up quite a bit then :)
Click to expand...

Yes, good idea, that would work, too. However, I don’t use Mac anymore, I switched to PC (transfering video from iPhone to PC is almost equally easy). Do you plan to develop a desktop application for Mac only, or PC app is also in the pipeline?

EDIT: I am also pretty sure that most people would prefer to watch the match analysis on the big computer screen.
 
swupnil

swupnil

Rookie
@Dragan A Mac app is a lot easier as Apple has made it so that with just a few extra lines of code, the iOS app can run on Mac. PC would require an entire app rewrite. We could potentially do it eventually, though, but the team needs to grow a lot larger.
 
J

jmnk

Hall of Fame
Dragan said:
Yes, good idea, that would work, too. However, I don’t use Mac anymore, I switched to PC (transfering video from iPhone to PC is almost equally easy). Do you plan to develop a desktop application for Mac only, or PC app is also in the pipeline?

EDIT: I am also pretty sure that most people would prefer to watch the match analysis on the big computer screen.
Click to expand...
@Dragan - when you say "(transfering video from iPhone to PC is almost equally easy)" - does it actually work for you all the time? I can indeed transfer most of the videos with no issues - but _any_ video over 4GB will not transfer from iPhone to a PC. Such video is not even detected by windows PC (meaning on windows PC it looks as if such video did not exist). Do you know the trick around that limitation?
 
J

jmnk

Hall of Fame
Demented said:
Just digging into your serve speed calculations a bit more. Are you reporting average ball speed(as measured by distance/time) or using the advance formula to figure out head velocity at impact?

import math # published numbers from a now dead link KNOWN_SERVE_SPEED= 120. KNOWN_SPEED_AT_BOUNCE= 87.0 DISTANCE_TRAVELED=60. # A constant for formula: K= math.log(120./87)/60 def init_speed(nf, d=60, fps=29.97): speed = ( math.exp(K*d) - 1) / (5280.*K * nf) * fps * 3600 return round(speed*100)/100 def avg_speed(nf, d=60, fps=29.97): speed = d / (nf / fps ) * (3600./5280.) return round(speed*100)/100

I book marked a serve speed calculator a few years ago that used known ball serve velocity from pro match radar as compared to final velocity at bounce and built this formula. I've since then verified it using Playsight.
Click to expand...
@Demented - if I may ask, what is that 'dead link' you got that formula from?
 
D

Deleted member 369227

Guest
jmnk said:
@Dragan - when you say "(transfering video from iPhone to PC is almost equally easy)" - does it actually work for you all the time? I can indeed transfer most of the videos with no issues - but _any_ video over 4GB will not transfer from iPhone to a PC. Such video is not even detected by windows PC (meaning on windows PC it looks as if such video did not exist). Do you know the trick around that limitation?
Click to expand...

I don’t recall ever having an issue with copying large videos from iPhone and iPad. I remember doing a screen recording on the iPad to grab a movie for my kids, and such videos were huge in size before reencoding in Handbrake App. For test, I will try recording a very long 4K@60fps video and will let you know how it went.

By the way, what is your Windows version and what is the file format of the destionation disk where you tried to copy that large video?

EDIT:

I just copied a long 4K video file (1:04:09) with a size of 5.23GB from iPad Pro to my PC laptop and everything went fine.

You might want to set this option on your iPhone and restart the device (it tells your iPhone not to convert videos to a "compatible" format before transferring to PC, but to copy it as-is):

yjEPFmz.jpg
 
Last edited by a moderator:
J

jmnk

Hall of Fame
Dragan said:
I don’t recall ever having an issue with copying large videos from iPhone and iPad. I remember doing a screen recording on the iPad to grab a movie for my kids, and such videos were huge in size before reencoding in Handbrake App. For test, I will try recording a very long 4K@60fps video and will let you know how it went.

By the way, what is your Windows version and what is the file format of the destionation disk where you tried to copy that large video?

EDIT:

I just copied a long 4K video file (1:04:09) with a size of 5.23GB from iPad Pro to my PC laptop and everything went fine.

You might want to set this option on your iPhone and restart the device (it tells your iPhone not to convert videos to a "compatible" format before transferring to PC, but to copy it as-is):

yjEPFmz.jpg
Click to expand...
I have the above setting on the iPhone Tried with 'Automatic' as well. Still no go. I have windows 10 with NTFS file format. Whether I use Windoes Photo app, or just plain explorer - Windows PC does not seem to recognize there's a video file on the iPhone if that file is over 4GB. Could you try with iPhone and not iPad?
 
D

Deleted member 369227

Guest
@jmnk

Sure, I use Android phone and iPad, but will try the same test with my wife's iPhone 11 Pro.

UPDATE:

I confirm that large files transfer from iPhone works equally fine. Just finished recording a huge 4K@60 video (38.2GB in size) and transferred it from my wife's iPhone 11 Pro to PC without a problem. You may try to update your iTunes (or even better uninstall and reinstall it) - drivers for iPhone may be outdated.
 
Last edited by a moderator:
J

jmnk

Hall of Fame
Dragan said:
@jmnk

Sure, I use Android phone and iPad, but will try the same test with my wife's iPhone 11 Pro.

UPDATE:

I confirm that large files transfer from iPhone works equally fine. Just finished recording a huge 4K@60 video (38.2GB in size) and transferred it from my wife's iPhone 11 Pro to PC without a problem. You may try to update your iTunes (or even better uninstall and reinstall it) - drivers for iPhone may be outdated.
Click to expand...
ok, so you use iTunes on PC to transfer videos from Apple devices? Because I do not. I'm trying to use windows Photo app, or plain explorer.
 
D

Deleted member 369227

Guest
jmnk said:
ok, so you use iTunes on PC to transfer videos from Apple devices? Because I do not. I'm trying to use windows Photo app, or plain explorer.
Click to expand...

No, I don’t use iTunes, just Windows Explorer. But I believe iTunes installs a necessary driver for iPhone.
 
D

Demented

Semi-Pro
Hey, if I was going to buy an apple device to run this app, what would be the best option for me? I'm assuming you've tested all the various combinations. Which device(outdide of the full size ipad) handles the recording/processing the best and can last the longest? I also have(from using the In.Out device) a portable solar panel that can produce like 15 watts.. any idea how much juice would be necessary to make this work?
 
swupnil

swupnil

Rookie
@Demented I would recommend an iOS device with built-in ultra wide angle lens so that you can see the entire court. So that would be iPhone 11 Pro Max if you’re trying to maximize battery life but don’t want an iPad. You might also just want to wait a couple weeks as the new 12 Pro Max will be the same price but likely more power efficient, with a better camera etc.
 
Last edited:
D

Demented

Semi-Pro
Do you make use of higher resolution recording or is it fixed to 1080p? The reason I was shying away from an Ipad full is due to the size and difficulty of hanging it on the fence. I was considering an Ipad Mini. Now I'm lookin at an Iphone SE since it has the wide angle camera.
 
Last edited:
swupnil

swupnil

Rookie
@Demented the iPhone SE has a standard camera but not the secondary “ultra wide” lens (sorry I mistyped originally). Basically I would recommend any iOS device made in 2019 or later that has two or more cameras :)

We don’t make use of the higher res recordings yet. It already takes quite a long time to back up the videos with the current resolution and the footage is downsized quite a bit for the A.I. to run efficiently.

If you like the size and price point of the SE, you should wait for the new iPhones next month. There will be a new smaller full screen iPhone around the same size as the SE that has dual cameras (wide + ultra wide) with even better processing than an iPhone 11.
 
J

jmnk

Hall of Fame
Demented said:
Just digging into your serve speed calculations a bit more. Are you reporting average ball speed(as measured by distance/time) or using the advance formula to figure out head velocity at impact?

import math # published numbers from a now dead link KNOWN_SERVE_SPEED= 120. KNOWN_SPEED_AT_BOUNCE= 87.0 DISTANCE_TRAVELED=60. # A constant for formula: K= math.log(120./87)/60 def init_speed(nf, d=60, fps=29.97): speed = ( math.exp(K*d) - 1) / (5280.*K * nf) * fps * 3600 return round(speed*100)/100 def avg_speed(nf, d=60, fps=29.97): speed = d / (nf / fps ) * (3600./5280.) return round(speed*100)/100

I book marked a serve speed calculator a few years ago that used known ball serve velocity from pro match radar as compared to final velocity at bounce and built this formula. I've since then verified it using Playsight.
Click to expand...
bump - @Demented if I may ask, what is that 'dead link' you got that formula from?
 
D

Demented

Semi-Pro
Sorry, the dead link is in the comments of the formula on the website with the calculator. I have no idea where it came from. I went back and searched for threads on the donthireddy.us calculator and read your posts on it. I experimented heavily with different combinations to make that formula fairly accurate(let's say within 3% margin of error). I upgraded to a 60 FPS wide angle camera and mounted it 10 feet above the court at the center line. I also took measurements of the exact distances to various places in the box. My hard first serve would average between 98 and 103 using the frame calculations and then I got a Playsight membership and could see it every time. My range on playsight was even more narrow, 99 to 101. If I threw my arm out of the socket and really pushed I could hit 108. My lazy serve was 93. Incase anyone is wondering, dead practice balls will rob you of 3-5 mph of velocity and cold ambient conditions will take away another 3-5.
 
Last edited:
Nostradamus

Nostradamus

Bionic Poster
Badmrfrosty said:
Just got an email and ordered this.

Swing Vision

It claims you can use just your iPhone to review line calls and get a detailed analysis of your play. Like automated shot tracking and match tracking. It would be cool to have accurate data on shot placement, speed, spin, etc... I don't have a PlaySight near me or I would definitely go play there. It's supposed to do all that just from Iphone.

I'm super skeptical as these things always tend to over-promise and underdeliver, but said full refund if after 2 weeks you don't like it. I'm going to try it out as if it works like promises killer app, but have a feeling it is going to be buggy and not accurate like the other data tracking tech nerd tennis stuff I've bought.

Anyone know about this or used the apple watch app they have? Anyone use that basketball app steve nash is pushing? Seems like a similar concept.

iphonex-combined-play.png
Click to expand...
How does it make line calls again ? and is it instant review ? how fast is it?
 
D

Demented

Semi-Pro
I noticed 7.5 came out yesterday. What improvements did you release in this build? I'm still trying to decide what the cheapest possible route is for me to get an apple device(I hate giving them money).
 
derick232

derick232

Rookie
Demented said:
I noticed 7.5 came out yesterday. What improvements did you release in this build? I'm still trying to decide what the cheapest possible route is for me to get an apple device(I hate giving them money).
Click to expand...
The update summery said that live tracking is now available for Iphone XR, 11, and several others. I plan on testing this out on Monday (Do you have to have an apple watch for this?). There was also an update to player selection when adding an opponent to a practice match to make it more user friendly. Again I'll check this out on Monday. Other than that I believe it was just bug fixes.
 
D

Demented

Semi-Pro
New version dropped yesterday, big enhancements. I'm going to try it this afternoon. I'd like to hear about anyone's experience using either an Ipad Mini 5 or an Iphone SE to run this. I don't use apple products so I plan on buying the cheapest option that will still do the job. The Ipad Mini obviously has the advantage of a huge battery but the camera is somewhat suspect. The SE seems to have a standard camera equivalent to the XR/XS but the battery is tiny. @swupnil, Does the chipset make a difference in performance?
 
D

Demented

Semi-Pro
I did a brief 10 minute test this afternoon and the upgraded analysis is much much better than it was. Kudos to @swupnil for producing this at this price point. He basically just destroyed everything else....

I did have a question, I used my buddies XS Max to do the recording and it made me analyze the video after we finished the hitting session. What level of phone is required for real time analysis now?
 
swupnil

swupnil

Rookie
@Demented thanks for the kind words! Still a ways to go but we’re constantly improving.

Basically Apple A12 chip and newer is supported for real time analysis. There have been some bugs where sometimes real-time analysis doesn’t work on iPhone XR/XS (or SE 2020) that we’re trying to resolve.

Currently the best way to guarantee real-time analysis is to force quit all other open apps. Obviously not ideal but we’ll get this working reliably soon!
 
derick232

derick232

Rookie
Hey @swupnil , do you think this could ever be used for other sports like volleyball? I play a lot of tennis and volleyball and just thought about how useful it could be in volleyball as well. Obviously some stats and features wouldn't work, but things like cutting the point up would be nice, and looking as specific shots like serves, sets, and kills. Any plans to expand the market?
 
derick232

derick232

Rookie
@swupnil I was just thinking about the types of overlays done on the ATP during matches. One that is very interesting and can be helpful to understand serving is serve toss placement. I don't know the difficulty of it but if you're already tracking the ball could you create an overlay for all your ball tosses? Knowing if you are tossing to far forward, back etc would be very beneficial to many players to become more consistent.
 
N

NeilonNeil

New User
Congrats to Swupnil on the SwingVision app and on all the features so far. Looking forward to improvements. Just wanted to say that I made a chain link fence phone holder that worked perfectly for $5 and use a iPhone 7 Plus with post match analysis. I haven’t figured out work around yet for the battery usage and storage size limitations but I have some ideas. I can post a picture of the chain link phone holder if there’s interest.
 
derick232

derick232

Rookie
NeilonNeil said:
Congrats to Swupnil on the SwingVision app and on all the features so far. Looking forward to improvements. Just wanted to say that I made a chain link fence phone holder that worked perfectly for $5 and use a iPhone 7 Plus with post match analysis. I haven’t figured out work around yet for the battery usage and storage size limitations but I have some ideas. I can post a picture of the chain link phone holder if there’s interest.
Click to expand...
Yes, I'd like to see what you've come up with. I'm actually working on designing one for my iphone 11 that I will be 3D printing. I'm thinking about making it compatible with GoPro mounts to make it more universal.
 
SonRK

SonRK

Semi-Pro
I just downloaded this app after recording footage of a session today and it doesn't seem like I can upload footage, I have to actually record with the app... damn!

Hopefully I can squeeze in another session before my trial is up
 
derick232

derick232

Rookie
SonRK said:
I just downloaded this app after recording footage of a session today and it doesn't seem like I can upload footage, I have to actually record with the app... damn!

Hopefully I can squeeze in another session before my trial is up
Click to expand...
Yes, you do all the setup and recording in the app. Though being able to upload footage would be a great feature that maybe they'll implement at some point.
 
N

NeilonNeil

New User
derick232 said:
Yes, I'd like to see what you've come up with. I'm actually working on designing one for my iphone 11 that I will be 3D printing. I'm thinking about making it compatible with GoPro mounts to make it more universal.
Click to expand...
Hi Derick and to the community,
Here’s pictures of my fence mount, stays flat on the fence so no worries about running into it, made from a short piece of pex plumbing pipe with 2 holes drilled in the side and two pieces of electrical wire, something every home build will have left over so talk to your local contractor and you are in business. Bought the selfie stick on the local used website for 5$. The handle fits perfectly in the pipe. (I can’t see how to add the picture here yet but will keep trying).
 
D

Demented

Semi-Pro
Another cheap option is to buy a selfie stick and add a hook to it. I bought a 19 dollar one off of amazon, stuck 2 Hercules hooks through the swivel joint at the top and I reach up and hook it to the top of the fence. I added an elastic cord to the end with a clip and I tension it to the fence below the stick to keep it from being able to move.
 
P

PaulG

New User
Hi swupnil,

Any word on the Android version? Is there a beta I can sign up for? Would hate to have to get an Iphone just to use this app.

Thanks,
 
You must log in or register to reply here.

Similar threads

D
problems with Swing Vision video playback
Replies
3
Views
577
waterpro
W
A
Swing Vision: great app, appalling customer support
Replies
1
Views
239
alaindesilets
A
ruhil
Using Maps and Data Visualization to Understand Tennis
Replies
0
Views
474
ruhil
ruhil
N
how to best measure serve speed, and compare to pros...
Replies
28
Views
5K
Injured Again
I
S
Electronic Line Calls
Replies
2
Views
747
West Coast Ace
West Coast Ace
Top