Home/News | About | Download | Documentation | Forum | Bug Reports | Contact | Donations | Consulting | Projects | Legal | Security | FATE


This forum has not been maintained for a long time and will probably get deleted in the near future.
For faster responses to your questions, please use StackOverflow instead and tag your questions with "FFmpeg".
If you need a backup of the posts from this forum, please contact me directly.

Difficulty with offsets on multi-stacked videos

Discussions about problems encountered using ffmpeg.
Forum rules
FFmpeg is in a state of perpetual development and we do not currently have the person power to backport fixes to official releases. As such, if you wish to query or report a bug, you must try with the latest development branch revision of FFmpeg to confirm the issue still exists. The latest FFmpeg can be downloaded from here: http://www.ffmpeg.org/download.html

Difficulty with offsets on multi-stacked videos

Postby dean99 » Thu Feb 09, 2017 3:21 pm

Hi everyone,

First off, thanks for an amazing product. I've been having a lot of fun learning and playing with ffmpeg. Powerful tool!

My end goal is to get 4 .webm videos (VP8) merged into a single, tiled, .webm video. Each of the videos has a slightly different start point. So video 1 might start playing immediately, video 2 (in top-right) might start playing 10 seconds later, video 3 (bottom left) another 10 seconds after that and finally video 4 another 10 seconds after that.

My command line implementation does seem to get close, but the start times are not aligning correctly. I'm not sure why this is. Additionally, I get a lot of screen flicker in playback, almost like my nullsrc background is actually a flickering video (lots of green lines and green blocks).

Here is my current implementation (I have added whitespace with tabs and newlines for clarity):-

Code: Select all
ffmpeg -i /srv/raw-video/76f853ae-795ba190/a.webm
            -i /srv/raw-video/76f853ae-795ba190/b.webm
            -i /srv/raw-video/76f853ae-795ba190/c.webm
            -i /srv/raw-video/76f853ae-795ba190/d.webm
-filter_complex
"nullsrc=size=640x480 [base];
   [0:v] setpts=PTS-STARTPTS, scale=320x240 [upperleft];
   [1:v] setpts=PTS+8/TB, scale=320x240 [upperright];
   [2:v] setpts=PTS+15/TB, scale=320x240 [lowerleft];
   [3:v] setpts=PTS+22/TB, scale=320x240 [lowerright];
[base][upperleft] overlay=shortest=0 [tmp1];
[tmp1][upperright] overlay=shortest=0:x=320 [tmp2];
[tmp2][lowerleft] overlay=shortest=0:y=240 [tmp3];
[tmp3][lowerright] overlay=shortest=0:x=320:y=240"


There are no errors on output.

I got the original concept for this from here:-

< edit - sorry as a new member it seems I can't post links, but if you google "Create a mosaic out of several input videos – FFmpeg" you'll find it >

.. and then reviewed the API docs to learn about setpts. My understanding is that my "setpts=PTS+x/TB" should effect an offset of x seconds.

So to summarise my two problems:-

1. setpts=PTS+x/TB does not seem to create an accurate offset of x seconds
2. I get this strange video1/green lines/green box flickering in what I was expecting to be a black background.

If anyone could shed any light I'd be grateful!

Thank you.
dean99
 
Posts: 11
Joined: Thu Feb 09, 2017 3:01 pm

Re: Difficulty with offsets on multi-stacked videos

Postby mulvya » Thu Feb 09, 2017 3:49 pm

Strictly speaking, nullsrc returns frames with value 0. In YUV colorspace, that's a shade of green. Black would be (0,128,128). For black, use color filter.

Since you are testing with the same input via 4 separate feeds, I suspect ffmpeg is dropping frames. Try with different videos, or copy and rename the file thrice. When shifting TS, do it relative to PTS-STARTPTS i.e. setpts=PTS-STARTPTS+8/TB.

The faster way to do the stacking is to use the dedicated stack filters.

Code: Select all
[upperleft][upperright]hstack[top];
[lowerleft][lowerright]hstack[bottom];
[top][bottom]vstack[tiled]
mulvya
 
Posts: 570
Joined: Mon Oct 26, 2015 6:58 am

Re: Difficulty with offsets on multi-stacked videos

Postby dean99 » Thu Feb 09, 2017 5:31 pm

Thank you mulvya for the reply.

In the first instance I thought I'd try the dedicated stack filters as you suggest, but I'm struggling with that. Current code:-

Code: Select all
ffmpeg -i /srv/raw-video/e727da41-5d9c2413/a.webm -i /srv/raw-video/e727da41-5d9c2413/b.webm -i /srv/raw-video/e727da41-5d9c2413/c.webm -i /srv/raw-video/e727da41-5d9c2413/d.webm
-filter_complex
"nullsrc=size=640x480 [base];
[0:v] setpts=PTS-STARTPTS, scale=320x240 [upperleft];
[1:v] setpts=PTS-STARTPTS+4/TB, scale=320x240 [upperright];
[2:v] setpts=PTS-STARTPTS+11/TB, scale=320x240 [lowerleft];
[3:v] setpts=PTS-STARTPTS+17/TB, scale=320x240 [lowerright];
[base][upperleft] overlay=shortest=0 [tmp1];
[upperleft][upperright]hstack[top];
[lowerleft][lowerright]hstack[bottom];
[top][bottom]vstack[tiled]
-c:v vp8 output.webm


Produces the error:-

Code: Select all
Too many inputs specified for the "hstack" filter.


... which is strange as I know I can have two inputs for this filter.

Any thoughts?

Black would be (0,128,128). For black, use color filter.


Ah, that makes perfect sense and explains the green. I can't find anything in the API docs when I search "color filter" though - can you give me an example of how to use it?

Thank you again.
dean99
 
Posts: 11
Joined: Thu Feb 09, 2017 3:01 pm

Re: Difficulty with offsets on multi-stacked videos

Postby mulvya » Thu Feb 09, 2017 5:53 pm

Try

Code: Select all
ffmpeg -i /srv/raw-video/e727da41-5d9c2413/a.webm -i /srv/raw-video/e727da41-5d9c2413/b.webm -i /srv/raw-video/e727da41-5d9c2413/c.webm -i /srv/raw-video/e727da41-5d9c2413/d.webm
-filter_complex
"color=black:size=640x480 [base];
[0:v] setpts=PTS-STARTPTS, scale=320x240 [upperleft];
[1:v] setpts=PTS-STARTPTS+4/TB, scale=320x240 [upperright];
[2:v] setpts=PTS-STARTPTS+11/TB, scale=320x240 [lowerleft];
[3:v] setpts=PTS-STARTPTS+17/TB, scale=320x240 [lowerright];
[base][upperleft] overlay=shortest=0 [tmp1];
[tmp1][upperright] overlay=shortest=0:x=320 [tmp2];
[tmp2][lowerleft] overlay=shortest=0:y=240 [tmp3];
[tmp3][lowerright] overlay=shortest=0:x=320:y=240"
-c:v vp8 output.webm


The thing with the stack filters is that they will freeze on the first frame of a stream till it starts and freeze on the last frame after it ends. So, I believe overlay route is required here.
mulvya
 
Posts: 570
Joined: Mon Oct 26, 2015 6:58 am

Re: Difficulty with offsets on multi-stacked videos

Postby dean99 » Thu Feb 09, 2017 7:41 pm

Thank you so much - it's so close now :)

I'm currently stuck in a loop somewhere as ffmpeg never exits (these are files on disk not live streams). So I need to work out how to stop it when the last video ends.

Is that you you were referring to with overlay route?

Interestingly, now with the timing, all are in sync other than the first video, which is behind the others by about 2 seconds.
dean99
 
Posts: 11
Joined: Thu Feb 09, 2017 3:01 pm

Re: Difficulty with offsets on multi-stacked videos

Postby mulvya » Thu Feb 09, 2017 8:06 pm

The first overlay should have shortest=1.
mulvya
 
Posts: 570
Joined: Mon Oct 26, 2015 6:58 am

Re: Difficulty with offsets on multi-stacked videos

Postby dean99 » Thu Feb 09, 2017 9:30 pm

You're a superstar, thank you very much for your help.

I have a timing issue still, but looking much better and I can play with it and do some more reading and testing as per your suggestions, like using the same file three times.
dean99
 
Posts: 11
Joined: Thu Feb 09, 2017 3:01 pm

Re: Difficulty with offsets on multi-stacked videos

Postby dean99 » Sun Feb 12, 2017 1:42 pm

I have almost my entire application working thanks to your help.

I have one further issue I wonder if you could help point me in the right direction with.

As well as the 4 video block I need to create a 3 video block, to look like this:-

Code: Select all
+————————————————————-+————————————————————-+
|                     |                     |
|                     |                     |
|                     |                     |
|          1          |           2         |
|                     |                     |
|                     |                     |
+——————————+-————————-+——————————+——————————+
           |                     |
           |                     |
           |                     |
           |          3          |
           |                     |
           |                     |
           +————————————————————-+


I am trying to do that with:-

Code: Select all
 ffmpeg -i /srv/raw-video/f7e90022-6ef36c13/0.webm -i /srv/raw-video/f7e90022-6ef36c13/1.webm -i /srv/raw-video/f7e90022-6ef36c13/2.webm
-filter_complex "
[0:v]setpts=PTS-STARTPTS[left];
[1:v]setpts=PTS-STARTPTS+2/TB[right];
[2:v]setpts=PTS-STARTPTS+9/TB[lower];
[left][right]hstack=inputs=2[v][lower];[a:0][1:a][2:a]amerge[a]" -map "[v]" -map "[a]"
-ac 3 /srv/raw-video/f7e90022-6ef36c13/f7e90022-6ef36c13.webm -loglevel error
[code]

... and I get the error:-

[code]
No output pad can be associated to link label 'lower'.
Error initializing complex filters.
Invalid argument


... which I don't really follow.

Is there a way to combine both hstack and vstack with an odd number of video inputs like this?

Thank you again.
dean99
 
Posts: 11
Joined: Thu Feb 09, 2017 3:01 pm

Re: Difficulty with offsets on multi-stacked videos

Postby mulvya » Sun Feb 12, 2017 2:10 pm

The stack filters require the dimension of the inputs along which they are joined to be the same i.e. hstack filters require the height of the inputs to be the same, and vstack the width. What is needed here is for the lower video to be padded so that its width is the same as that of the combined top layer.

(In any case, I don't see the use of the vstack filter in your command. Also, the stack filters only produce one output.)

Assuming all videos have the same dimensions, the command should be

Code: Select all
 ffmpeg -i /srv/raw-video/f7e90022-6ef36c13/0.webm -i /srv/raw-video/f7e90022-6ef36c13/1.webm -i /srv/raw-video/f7e90022-6ef36c13/2.webm
-filter_complex "
[0:v]setpts=PTS-STARTPTS[left];
[1:v]setpts=PTS-STARTPTS+2/TB[right];
[2:v]setpts=PTS-STARTPTS+9/TB,pad=2*iw:ih:iw/2:0[lower];
[left][right]hstack=inputs=2[upper];[upper][lower]vstack[v];[0:a][1:a][2:a]amerge[a]" -map "[v]" -map "[a]"
-ac 3 /srv/raw-video/f7e90022-6ef36c13/f7e90022-6ef36c13.webm -loglevel error
mulvya
 
Posts: 570
Joined: Mon Oct 26, 2015 6:58 am

Re: Difficulty with offsets on multi-stacked videos

Postby dean99 » Sun Feb 12, 2017 3:13 pm

Thanks for the help again mulvya.

ffmpeg doesn't seem happy with the audio filter part (the intention was to mix all three audio streams):-

Code: Select all
[AVFilterGraph @ 0x185a0e0] Too many inputs specified for the "amerge" filter.
Error initializing complex filters.
Invalid argument


Is this because ffmpeg now sees this as only 2 audio streams, given 0+1 are already merged into a hstack ?
dean99
 
Posts: 11
Joined: Thu Feb 09, 2017 3:01 pm

Next

Return to ffmpeg

Who is online

Users browsing this forum: Bing [Bot], Google [Bot] and 8 guests