Imagine there's an app that auto-trims your match videos, how should it work?

psed

New User
I’ve been recording my matches for a while to try and improve my game. But if I'm being honest, I realized I rarely actually watched the footage back. It was just too painful to manually scrub through hours of dead spots just to find the actual points. (I'm going to pretend that me being bad at tennis is not the reason why it's painful to watch. Ok maybe it's the primary reason. But anyways.) I actually tried editing my match twice using Davinci. It took a huge amount of time and effort. Kudos to those who do this regularly. I looked into some apps but I have no desire to switch to iPhone just for this.

Since I'm in a related field, I was curious if I could automate this using computer vision. And then my elbow issue flared up so I thought I'd hit my keyboard instead of my balls.

Fast forward, I spent a good amount of evenings and weekends building models to recognize the difference between "play" and "non-play" states in tennis match videos. It actually works surprisingly well! It takes a raw 1-hour recording from my phone and automatically chops it down to a tight 25-30 minute video of just the rallies.

So... imagine we somehow get YOU to use this model. We'll need something (like an app, website, etc) to ingest your video, process it, and send it back to you.

How should it work?

- Do you want all the processing locally or on cloud?
  • locally on your phone: will be slow and eat your battery. This will be way more difficult for me to make.
  • on the cloud: needs strong wifi for upload
- Do you want this as a website or a mobile app?
  • Website: can work on any device
  • Mobile app: might work a bit smoother? depends on other factors
- What kind of deliverable do you want?
  • A (edited) video of your full match?
  • Just the highlight reels? (auto-selected or manually by yourself)
  • Just one rally per video file?
  • A premiere or davinci plugin? (I don't know if this is a thing at all)
- Willing to pay for this? GPU servers are expensive and I won't be able to cover the cost just with ads.


ps. Do you have a match recording that you're too lazy to edit manually? Send that to me and I'll auto-edit it for you. I get to test this thing, you get an edited video. Maybe post on youtube and DM me the link.
 
Great idea.
I'd say cloud.
I like the covneience of app, but I prefer watching vids on a larger computer screen so lean towards website.
Full match edition would be best with maybe option for highlights or filters.
 
Great idea.
I'd say cloud.
I like the covneience of app, but I prefer watching vids on a larger computer screen so lean towards website.
Full match edition would be best with maybe option for highlights or filters.

Thanks so much for your input!
Your comment made me realize that being a website (vs a mobile app) means access to bigger screens.
Full match priority -- noted.

Thanks again!
 
Swingvision does this. Dead spots are removed from matches and practice play, while shot placement and speed are projected in the corner. Check out their website for details.

 
Swingvision does this. Dead spots are removed from matches and practice play, while shot placement and speed are projected in the corner. Check out their website for details.

My question is if I use the camera from the side will the video still scrubb properly or will it be way off since it is used to a behind the court view.
If I use it in this fashion, I understand why speed and placement wouldn't register properly. Just wondering if the scrubbing will.
 
Swingvision does this. Dead spots are removed from matches and practice play, while shot placement and speed are projected in the corner. Check out their website for details.


Thanks for your reply. To clarify, I'm trying to brainstorm something that (1) works on any device and (2) has a user interface & workflow that are optimized for video editing.
Huge respect to swingvision, but I think a specialty tool for this video editing task that works on all devices might be neat. Just wondering what people think the tool should look like.
 
My question is if I use the camera from the side will the video still scrubb properly or will it be way off since it is used to a behind the court view.
If I use it in this fashion, I understand why speed and placement wouldn't register properly. Just wondering if the scrubbing will.

Haven't run this myself yet, but I suspect my model will grab something but not everything from this sideline view video. So you'll need quite a lot of manual adjustments, defeating the purpose of using a tool like this.

Computers are quite good at understanding the exact thing they're looking at, but very poor at understanding the context, unfortunately.
I guess one will need to build a model specifically for this (or a model specifically built to work with any video angle).

Right now I'm more focused on the baseline view (i.e. view from the player's back) because it's more common and has more information (this angle captures both players and most of the ball trajectory).
But someday I want to tackle this.

It's a fun thought experiment! Thanks for your input.
 
I’ve been recording my matches for a while to try and improve my game. But if I'm being honest, I realized I rarely actually watched the footage back. It was just too painful to manually scrub through hours of dead spots just to find the actual points. (I'm going to pretend that me being bad at tennis is not the reason why it's painful to watch. Ok maybe it's the primary reason. But anyways.) I actually tried editing my match twice using Davinci. It took a huge amount of time and effort. Kudos to those who do this regularly. I looked into some apps but I have no desire to switch to iPhone just for this.

Since I'm in a related field, I was curious if I could automate this using computer vision. And then my elbow issue flared up so I thought I'd hit my keyboard instead of my balls.

Fast forward, I spent a good amount of evenings and weekends building models to recognize the difference between "play" and "non-play" states in tennis match videos. It actually works surprisingly well! It takes a raw 1-hour recording from my phone and automatically chops it down to a tight 25-30 minute video of just the rallies.

So... imagine we somehow get YOU to use this model. We'll need something (like an app, website, etc) to ingest your video, process it, and send it back to you.

How should it work?

- Do you want all the processing locally or on cloud?
  • locally on your phone: will be slow and eat your battery. This will be way more difficult for me to make.
  • on the cloud: needs strong wifi for upload
- Do you want this as a website or a mobile app?
  • Website: can work on any device
  • Mobile app: might work a bit smoother? depends on other factors
- What kind of deliverable do you want?
  • A (edited) video of your full match?
  • Just the highlight reels? (auto-selected or manually by yourself)
  • Just one rally per video file?
  • A premiere or davinci plugin? (I don't know if this is a thing at all)
- Willing to pay for this? GPU servers are expensive and I won't be able to cover the cost just with ads.


ps. Do you have a match recording that you're too lazy to edit manually? Send that to me and I'll auto-edit it for you. I get to test this thing, you get an edited video. Maybe post on youtube and DM me the link.
1. Cloud is better, but how would you handle storage needs?
2. mobile app would be better because the user experience is more seamless (recording on an iPhone and uploading directly via the app). website browsers on phones are very clunky and annoying.
3. a single trimmed mp4 video file (that the user can then download for personal use/backup)
4. Yes, would be willing to pay, if it can auto overlay the score after each point and performs as well as Swing-Vision in that regard.
 
Hey everyone, quick update on this. First off, sorry for the radio silence. My daughter decided to come into this world a bit unexpectedly early, so things have been quite crazy! Surprisingly enough, I managed to build the prototype we talked about.

I personally think it's useful enough to share, even though it's not perfect yet. https://www.rallyflowapp.com
It automates the trimming process we discussed so you get just the highlights/rallies.

If you want to try it, please use this special link I made for this thread: https://www.rallyflowapp.com/en/signup/special?t=talktennis

My app does some real heavy lifting (video processing), and the server costs will snowball. I had to add a "usage fee" mechanism hoping to not go bankrupt from this.
HOWEVER, I am not asking you to pay to test this. The link above gives you free credits to start. (Actually, you can't pay me even if you wanted to because the payment page is not even built yet haha).

If you use up the credits and want to test more, just DM me and I'll manually add more for you.

Let me know if it breaks with your specific camera setup, court texture, lighting, etc. I'm still tweaking the model, so go easy on me.
 
Back
Top