Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

How to manipulate videos on HDFS with FFMPEG ?

Highlighted

How to manipulate videos on HDFS with FFMPEG ?

Contributor

Hello everyone ,

I want to manipulate videos stored in HDP HDFS with the FFMPEG libraries and store results on a text file on HDFS or on Hbase, can you offer me ideas?

thanks for your help

4 REPLIES 4

Re: How to manipulate videos on HDFS with FFMPEG ?

Super Guru
@ELMEHDI SAOUDI

There is an example on github which involves working with both FFMPEG for data stored in Hadoop.

https://github.com/timlukins/pixelmor

I am not sure how processing with FFMPEG works but storing either in HDFS or HBase is pretty easy using standard Hadoop/HBase libraries. I would say start with simple HDFS and don't worry about HBase for now. If after doing first pass, you are not satisfied with HDFS, then post a question again and we'll help you with HBase. I just don't know how FFMPEG works. Hopefully, this link helps. If not, I hope someone who has worked with FFMPEG will help you.

Re: How to manipulate videos on HDFS with FFMPEG ?

Contributor

HI @mqureshi,

In your example the FFMPEG processing is local but personally I want it in hadoop.thank you

Re: How to manipulate videos on HDFS with FFMPEG ?

New Contributor

Have a look at the following:

Hadoop Streaming: https://hadoop.apache.org/docs/r2.7.2/hadoop-streaming/HadoopStreaming.html

You will want to use a custom inputreader capable of reading your input video in a meaningful way. e.g. See how XML does it for an idea because you can't just read video data 1 line at a time.

https://hadoop.apache.org/docs/r2.7.2/hadoop-streaming/HadoopStreaming.html#How_do_I_parse_XML_docum...

Depending on what you are doing you might be able to use ffmpeg to split the video into chunks using time parameters. e.g.

ffmpeg -i pipe:0 -ss 00:00:50.0 -codec copy -t 20 output.mp4

You would then assign 1 chunk to 1 mapper. So lots of little jobs each working on a different section of the video. Your reducer would then be ffmpeg to assemble all of them together into your final product.

See the following link for what is possible with ffmpeg

http://www.labnol.org/internet/useful-ffmpeg-commands/28490/

Also be sure to use pipe:0 and pipe:1 for parameters that deal with filenames as Hadoop streaming uses stdin and stdout.

Hopefully this gets you started

Re: How to manipulate videos on HDFS with FFMPEG ?

Contributor

Hello, Thanks for your reply, to clarify my objective of FFMPEG, I want to extract the motion vectors from a mp4 video, I can do this locally but I want to do this on hadoop. The command is:

ffmpeg -flags2 +export_mvs -i input.mp4 -vf codecview=mv=pf+bf+bb output.mp4

And then I modify the vf_codecview.c file to have the motion vector on a text file. For the moment my goal is to turn the command on hadoop and see its computing power on a video and then on my database of videos. @mqureshi @shooton

9799-vismv-pf.png