This was going to be a highminded post about fairy tales and morphology but instead i am going to record my trials and trivails of running ffmpeg on amazon's map-reduce.
So - what i am doing is adding a cached directory of ffmpeg to every map-reduce computer i use
#!/usr/bin/bash
./elastic-mapreduce --create --stream --alive --name $1 \
-c credentials.json \
--input s3n://13bit.videoout \
--mapper s3://13bit.scripts/compression.py \
--reduce s3://13bit.scripts/reduce.py \
--output s3n://13bit.videoout.ts \
--log-uri s3n://13bit.log \
--cache-archive s3n://13bit.scripts/ffmpeg.zip#ffmpeg
in order to build ffmpeg i started up a debian sqeeze box via aleatic ami (ami-daf615b3) http://alestic.com/. I then had to update apt-get (apt-get update), install git, download ffmpeg and related libraries and run the ffmpeg configure switch --prefix to set the install directory. I am now zipping it up and testing it on another instance of the squeeze debian - at which point i will start up the map reduce process again
i am also looking at boston terrier art on etsy (thanks to Jane Kim)
http://www.etsy.com/listing/37552859/boston-terrier-dog-art-original-painting
I am doing this while my boston terrier sleeps and farts next to me.
Showing posts with label ffmpeg. Show all posts
Showing posts with label ffmpeg. Show all posts
Monday, September 20, 2010
Wednesday, September 15, 2010
More Adventures with Map Reduce and the Cloud
I am compressing video in the cloud via amazon's EC2/mapreduce/s3/etc... I am using map/reduce for this although it is really not a reduce problem only a map problem and not even that - I just need to spawn a bunch of ffmpeg processes....
The problem is I cant map/reduce on a disk that has ffmpeg. So what I did was compiled map reduce on my machine - zipped it - uploaded it to s3 then added a bootstrap action to unzip and ffmpeg whenever the program starts... lets see how this works - I think I need to recompile on debian though - currently using fedora
As for the arabian nights - i have learned that the arabic oral tradition no longer exists because people sit around in cafes and watch tv instead of telling/listening to stories - ahh technology
The problem is I cant map/reduce on a disk that has ffmpeg. So what I did was compiled map reduce on my machine - zipped it - uploaded it to s3 then added a bootstrap action to unzip and ffmpeg whenever the program starts... lets see how this works - I think I need to recompile on debian though - currently using fedora
As for the arabian nights - i have learned that the arabic oral tradition no longer exists because people sit around in cafes and watch tv instead of telling/listening to stories - ahh technology
Wednesday, September 8, 2010
More EC2, FFMPEG and video streaming from the cloud
So today I am setting up the compression engine on EC2 to compress video out into 4 formats. I need to do this before I use the segmenter to divide up video out into segments that will then be recompiled on the fly via lexical searches...
Anyway I had a java program downloading my mov files - clips from video out- but it was taking to long. I dont want to have to download the file in order to convert it I want ffmpeg to convert it via http... luckily i found this on stack overload:
http://stackoverflow.com/questions/2677317/how-to-read-remote-video-on-amazon-s3-using-ffmpeg
evermind, I found an easy way to solve my problem.
I set up an amazon cloudfront download distribution pointing to my S3 bucket.
Via cloudfront the files are accessible with ffmpeg over http:
ffmpeg -i "http://subdomain.cloudfront.net/video.mp4" -ss 00:00:10 -vframes 1 -f image2 "image%03d.jpg"
Anyway I had a java program downloading my mov files - clips from video out- but it was taking to long. I dont want to have to download the file in order to convert it I want ffmpeg to convert it via http... luckily i found this on stack overload:
http://stackoverflow.com/questions/2677317/how-to-read-remote-video-on-amazon-s3-using-ffmpeg
evermind, I found an easy way to solve my problem.
I set up an amazon cloudfront download distribution pointing to my S3 bucket.
Via cloudfront the files are accessible with ffmpeg over http:
ffmpeg -i "http://subdomain.cloudfront.net/video.mp4" -ss 00:00:10 -vframes 1 -f image2 "image%03d.jpg"
Subscribe to:
Posts (Atom)