Swing Vision AI line calls and Analysis

ichaseballs

Professional
Thanks for the reply. Actually it is on a swingstick and it does have the full-court in view. I cropped it out just to keep the overlay with the names out so I didn't post them up online. In all of these rallies, the camera was completely static. I do work in the field of computer vision and it is surprising to me how poorly it is working in some of these situations and I can't tell if this is a bug of some sort impacting me or my device specifically. It also looks like they don't utilize sound cues at all to refine bounce timing which is surprising.

the swingstick or similar makes the angle tilted which bothers me.
i need the horizon to be straight horizontal
 

hamsterhuey

New User
Ah gotcha, yes that is annoying. Ironically, they need the elevated angle for best accuracy to better track lines and calls on the far-side.
 

ichaseballs

Professional
Ah gotcha, yes that is annoying. Ironically, they need the elevated angle for best accuracy to better track lines and calls on the far-side.

yep.
also indoors, the swingstick (on curtains) will shake and move slightly when the ball hits the curtain hard.
once that happens, the lines become much less accurate.
 

mptennis

Rookie
Does everyone else have SwingVision make egregiously bad line-calls or bounce location estimates fairly often, even for very easy balls that it should get right? I haven't been all that impressed with the quality of line-calling and I find it hard to believe that this is meeting metrics for automated line-calling. At the moment it seems way worse than well-intentioned players calling balls in/out from my experience. For the record, I am using a swing-stick with the top extended all the way for the highest vantage point and an iPhone SE 3rd gen.

Some examples attached
Fellow swing stick user here. Honestly I barely use the line-calling feature. I mainly use Swing Vision for reviewing videos for improvement, and the way it cuts out dead time is really valuable, in addition to the filters. I've found if I pay much attention to the line calling there are often big errors, but the majority of the time the ball is relatively close, probably within a foot. But 5% errors over 600 shots is 30 shots which is pretty significant.
 

hamsterhuey

New User
Fellow swing stick user here. Honestly I barely use the line-calling feature. I mainly use Swing Vision for reviewing videos for improvement, and the way it cuts out dead time is really valuable, in addition to the filters. I've found if I pay much attention to the line calling there are often big errors, but the majority of the time the ball is relatively close, probably within a foot. But 5% errors over 600 shots is 30 shots which is pretty significant.
Thanks. Yeah, I have typically overlooked it but last night's game had a lot of egregious misses to the point that it was significantly impacting the autoscoring... which was annoying as I wanted to share with my partners. I literally had to manually edit the winner of each point in our 3rd set tie-breaker to have it work. I'm usually fine with if the calls are reasonably close to where they are landing, but right now, like with the example above, they are often completely nonsensical at which point I can't trust any of the spray charts, and it even impacts the filtering in some cases when reviewing videos.

Agree that the biggest benefit is the auto-edited points, but even with that, sometimes the points are getting messed up when it makes egregious errors with the line-calling.
 

forzmr_b

Rookie
I feel like when auto-scoring made the full release, shot placement and line-calling accuracy dropped. It seems like there’s more instances of shot placements that are way off and when you edit the shot, typically the bounce timing is way off, throwing the placement and shot speed. In the past, line calling on the near side was hardly an issue, but now you get a minority of cases where the challenged call is wrong when you review the footage after (via the Apple Watch it was difficult to tell). And these cases don’t have obstructed views of the ball bounce or lines.

I’ve noticed that run around forehands almost always get classified as backhands now, when I didn’t notice it happening that frequently in the past too.

Consequently, I still prefer tracking points manually with the watch and have turned off shot placement, ball speed, shot type overlays when exporting since they are no longer accurate.
 

hamsterhuey

New User
Thanks. Yeah I reprocessed my video again and the shot placement / line-calling accuracy seems to be quite atrocious. It would be one thing if it was missing things that were within half a feet or so, but right now, a lot of the bounces are being marked 4-6 feet away from where they land. As a new subscriber, this has been quite disappointing. There is obviously still value in an edited down video, but right now, even there, I am seeing points not being split up correctly because the shot placement is so poor at times that their algorithms have no clue what the correct point duration was and how to appropriately clip the video for the point.
 

Falko83

New User
@swupnil , is there any possibility of making Swing Vision communicate/export to Strava? It is a pain that the activities I use Swing Vision aren't shown into Strava/there isn't any place where I can correctly track my heart rate during the match/activity.
 

AndrewUtz

Rookie
anyone know anything about the new app Play/Replay? not available in my country but it appears to do similar stuff to swingvision
 

Falko83

New User
anyone know anything about the new app Play/Replay? not available in my country but it appears to do similar stuff to swingvision

It looks like it's a different thing. Complete infancy, it's YT channel has only 1 subscriber, requires external sensors that are not your phone and do not film the session, is only a tool to provide stats, line calling and analysis. IMO the reason most people use swing vision is to have a better tool to actually see yourself playing, with the automated cuts. Swing Vision's stats and line calling are completely messed up - at least in my case, and that may be related to camera positioning, but that is not the reason why I use the app.
 

nyta2

Hall of Fame
@AndrewUtz the better line calling requires 4K video, which costs us 4x as much to upload & store in the cloud. Thus we have to charge for it.

If we can someday figure out a way to record at 4K but save the video at 1080p, we’ll gladly bring the better accuracy to Pro. We’re not trying to gatekeep better accuracy at all - just trying to be reasonable about the increased COGS on our end.
some random thoughts (regarding 4k as a means to improve accuracy)..
  • to reduce costs, can the localApp be smart enough to only upload snippets (around the lines?)
  • can their be tiered pricing that enabled 4k mode to be uploaded/processed
random thoughts regarding line calling accuracy
  • would be cool if there was a way to have 2-4 phones around the court sync'd together to form a more complete picture (and would motivate more folks to buy a subscription... presuming it's possilbe, and line calling was accurate, imagine if every tournament required competitors to have a subscription and carry around a tripod/fence mount....)
  • what if there was a mode to only track a single line (if it made the line calling algo easier)... imagine having 6 phones setup (2 baseline, 2 sidelines, 2 svc lines)... i'm sure tennis parents would buy it (so could probably have at least 4 per match)
 

happyloman

Professional
@swupnil Did the User interface change (for the worse) on Apple Watch ?

I had been using the Apple Watch for scoring matches but the score no longer shows up on Point+ mode the last couple of months. On version 11.6 (39.22)

Also, would love to be able to have more than just the "rally" and "Match" options show up on the Apple Watch ("Serve practice" is available only on the Iphone app but not on the Apple Watch? )
 

swupnil

Rookie
@hamsterhuey the screenshots you shared are quite surprising. We’ve been rigorously testing with USTA and the line calls (and shot placements) have been nearly perfect, certainly well better than a human.

In general, the app is optimized to accurately estimate “in” or “out” as accurately as possible. The actual 3D location within each region of the court is less accurate, but is significantly better in our testing compared to what you shared.

My only guess is that those additional short court lines on the court are messing things up and have low representation in our training set. USTA junior tournaments are never played on such courts and most of our customer base does not play on such courts.

Probably a data issue but thanks for reporting! If you can DM me a link to your session, we’d love to get it in the training set so that accuracy can improve.
 
Last edited:

swupnil

Rookie
@forzmr_b our accuracy metrics for in/out and 3d location are the highest they’ve ever been across a variety of courts where we’ve set up a 16 camera system to validate the accuracy. It would be super helpful to get session links from you so we can take a closer look (feel free to DM).

We’ve just come off a few junior tournaments with USTA where SwingVision provided line calling on 10+ courts and the accuracy was really solid.

I’m honestly very surprised to hear your feedback but would really like to look into this.
 

swupnil

Rookie
@nyta2 the line calling is all done on device so there’s no need to only “upload” content near the lines. The solution is basically what I mentioned before: process 4K but save in HD. This will likely only be possible on the very newest phones since downsizing every frame from 4K to HD in real-time at 60fps is super intensive.

We don’t have plans to support multiple cameras around the court or have cameras on specific lines. Over the last 5+ years we have seen other competitors do this. They have all come and gone. It’s just way too much friction and the market for that is just a very tiny handful of super users. It simply doesn’t scale and will never be a sustainable business model.
 

thecatch33

New User
@forzmr_b our accuracy metrics for in/out and 3d location are the highest they’ve ever been across a variety of courts where we’ve set up a 16 camera system to validate the accuracy. It would be super helpful to get session links from you so we can take a closer look (feel free to DM).

We’ve just come off a few junior tournaments with USTA where SwingVision provided line calling on 10+ courts and the accuracy was really solid.

I’m honestly very surprised to hear your feedback but would really like to look into this.
My experience as I have mentioned earlier in this thread is similar to @forzmr_b which is to say, in/out accuracy is not great. This is why I suggested adding an option to disable auto scoring, because the inaccuracy sometimes leads to scoring inconsistencies.

I think it’s frankly more surprising that you are surprised, as several other people in this thread (not just myself and @forzmr_b ) have commented that line calling is unreliable, and that seems to be the general feedback on Reddit, as well, at least from what I have seen.
 

swupnil

Rookie
@thecatch33 yea I guess there is a lot of variability out there that we haven't accounting for. hamsterhuey's videos have those other court lines on top while forzmr_b's videos are zoomed out, have curvature on the baseline (usually due to third-party wide angle lenses) and the session says "Suboptimal Setup" within the app.

If you're not seeing the Suboptimal Setup and still finding clear in/out issues, then yes I am very surprised. We have thousands of support tickets and any time there is inaccuracy, it's almost always that. When people setup as intended, the results have been really good. Hard to standardize for such a consumer product but we're doing our best to improve everywhere we can.

As to your Auto Scoring concerns, there is supposed to be a Revert Scoring button if you don't like the results. I will double check.
 

forzmr_b

Rookie
@thecatch33 yea I guess there is a lot of variability out there that we haven't accounting for. hamsterhuey's videos have those other court lines on top while forzmr_b's videos are zoomed out, have curvature on the baseline (usually due to third-party wide angle lenses) and the session says "Suboptimal Setup" within the app.

If you're not seeing the Suboptimal Setup and still finding clear in/out issues, then yes I am very surprised. We have thousands of support tickets and any time there is inaccuracy, it's almost always that. When people setup as intended, the results have been really good. Hard to standardize for such a consumer product but we're doing our best to improve everywhere we can.

As to your Auto Scoring concerns, there is supposed to be a Revert Scoring button if you don't like the results. I will double check.
@swupnil It isn’t easy to achieve the optimal setup. Which would be better, zooming out .8x without wide angle or zooming in 1.1x with wide angle?
 

swupnil

Rookie
@forzmr_b the best is to use the standard 1x with no third-party lens if that allows you to see all 4 corners of the singles court and even just a few pixels beyond the corners.

If you’re unable to see the court, I would first recommend 0.9x (or 0.8x zoom if absolutely necessary) if you have the newer ultra wide lens introduced in iPhone 13 pro / 14.

If your phone is older then that, then last resort is a Xenvo lens.
 

forzmr_b

Rookie
@forzmr_b the best is to use the standard 1x with no third-party lens if that allows you to see all 4 corners of the singles court and even just a few pixels beyond the corners.

If you’re unable to see the court, I would first recommend 0.9x (or 0.8x zoom if absolutely necessary) if you have the newer ultra wide lens introduced in iPhone 13 pro / 14.

If your phone is older then that, then last resort is a Xenvo lens.
I use an iPhone 13 mini so no in-built wide-angle lens. In the past, I had to zoom out to capture both singles sidelines. I’ll see if I can see the lines with 0.9x. If I can’t, would 0.8x w/o Xenvo be better or 1.1x with Xenvo? Whichever gets me closer “crop” of the 4 corners?
 

forzmr_b

Rookie
I use an iPhone 13 mini so no in-built wide-angle lens. In the past, I had to zoom out to capture both singles sidelines. I’ll see if I can see the lines with 0.9x. If I can’t, would 0.8x w/o Xenvo be better or 1.1x with Xenvo? Whichever gets me closer “crop” of the 4 corners?
Quick update. Without the Xenvo I have to rely on 0.8x zoom and results weren’t great. I also realised the next increment for zooming in was 1.5x so I didn’t bother trying to zoom in with the Xenvo attached. Guess I’ll have to make do with my Xenvo at 1x and accept the sub-optimal setup.
 
Top