3D 3K 12-bit RAW camera project

zcream

Why?

I have decided to shoot my next film in 3D. For this purpose, I can use a Panasonic Z10000 camera with 1080p @ 25P and 24P. The 4:2:2 lossless signal can be recorded with a BM Ultrastudio to dual Prores streams. However, 3D presents some challenges in post production. Upsizing 3D is a no-no as this may causes various violations in the 2 streams. Also, any jerkiness in 3D makes the viewer feel dizzy - a sensation similar to sea-sickness. The alternative is to shoot in a higher res and use a zoom and pan-scan in post to simulate movement. The HIT can be adjusted in post to add to the simulation of movement. http://realvision.ae/blog/2011/04/stereoscopic-3d-cinematography-zooming-in-3d/

How?

In another project, an inexpensive USB3 Flea3 FL3-U3-88S2C-C camera by Point Grey was used as a 4K RAW camera project.

A statement was made that using binning, 2K @ 60 fps RAW footage displayed minimal rolling shutter. This post indicated that 200 MBps is the limit that the Flea3 cameras have for bandwidth.

This camera can output 12-bit RAW at 24,25 fps at slightly above 3K. This allows for a resolution budget to allow for stabilizing and post zoom and pan,scan actions. Thus, I can keep my camera steady on a tripod while filming and worry about dolly moves in post. This allows for 3K in 12-bit RAW or a low rolling shutter 2K at 12-bit RAW.

More Details?

The lenses used in actual production will be fixed focal length as POV is hard to fix. I will attempt to connect an embedded i7 PC with 3x SSDs - 1 for the OS and 2 for recording each stream using uncompressed RAW. The gfx card does not make a difference - the key point is the CPU. A single recorder is preferred.

Some software options - COTS - Streampix with optional Cineform RAW - 1000/- + optional CF RAW license Norpix recorder Smallpix 2 with 2x SSDs - 2500/- + SSDs

My first attempt is to replicate what Soeren did in his thread and just use the supplied FlyCapture app.

Cesar Rubio has attempted this before.

However, this last approach was done lugging around a bulky PC. I wanted mine to be battery operated and portable.

The USB3 cameras can be used for anaglyph preview using Stereoscopic Player. The embedded PC can be controlled remotely using VNC or Remote Desktop.

Options for data rate with uncompressed RAW -

2048 x 1080 x 12bit, 60fps (199MB/s, binning, full sensor)
3120 x 1754 x 12bit, 25fps (205MB/s, windowed)
3200 x 1800 x 12bit, 24fps (207MB/s, windowed)
3584 x 2016 x 8bit, 25fps (181MB/s, windowed)
3728 x 2096 x 8bit, 24fps (188MB/s, windowed but only very slightly of course)

http://ns2.scarletuser.com/showthread.php?2043-Little-raw-cam-side-project&p=43776&viewfull=1#post43776

The camera has good detail and can look cinematic with the right grading. http://ns2.scarletuser.com/showthread.php?2043-Little-raw-cam-side-project&p=40070&viewfull=1#post40070

zcream

My first trial recorder will be the Retina Macbook Pro mid-2012 2.3 GHz i7. Soeren was able to record one stream using an embedded pico-ITX quad-core 1 GHz Via CPU. With the Macbook Pro, I can connect a dual SSD using Thunderbolt. The 2x USB3 streams can be used for 2x cameras. Its important to understand that the limitations of 200 MBps was related to the USB3 driver per camera.

So far, I am hoping that this is a limitation per USB3 connection, unrelated to the USB3 standard itself.

Two cameras can sustain 1.3 MP 120 FPS each, for a total of 340 MB/s, into a single $60 USB 3.0 interface card with no dropped data

http://ww2.ptgrey.com/USB3-resources

zcream

While the camera makes its way to me, I decided on the capture hardware. The last Macbook Pro with a DVD drive was the mid-2012 Macbook 9,1. I decided on the i7 model with High res 1680x1050 display. This has the advantage of being a standalone capture and edit station. It runs FCP7 and Davinci Resolve. The GT650m is also CUDA capable. The fastest version is a Core i7 Ivy Bridge 2.7GHz. I am going to use the Samsung 480GB or 960GB SSD in the SATA III capable DVD drive using a caddy. This SSD is capable of 400MBps write speeds. The 480GB version allows for a 3D uncompressed RAW 3K recording upto an hour. If more storage is needed, I can copy it to the standard HDD supplied in the MBP.

Vitaliy_Kiselev

Why use MacBook instead on much cheaper PC?

zcream

I have also decided to research options for compressed storage. Fast Compression - based in Moscow offers a CUDA JPEG encoder. JPEG offers 12-bit encoding options in 4:2:2 and 4:4:4.

The open source version with CPU encoder are available here - http://trac.osgeo.org/gdal/wiki/TIFF12BitJPEG http://www.gdal.org/frmt_jpeg.html

Uncompressed JPEG offers 3-5x compression. Even a 97% quality JPEG will offer 10x compression and be visually lossless. At 12-bit 4:2:2 the image quality will be good enough for painless keying while offering lower compression.

zcream

@Vitaliy_Kiselev Because my favorite cutting editor is FCP7. Second reason is that a Apple product is not that more expensive on the trailing edge of the tech curve. i.e. Last years tech. Also, it keeps its value better. I can use this for a couple of years and sell it for a decent price.

This also allows usage of the thunderbolt port for Blackmagic Ultrastudio 3D for lossless capture from Z10000 as well as grading with Resolve.

It allows for a flexible setup.

Vitaliy_Kiselev

My point is that in PC area exist powerful small computers. And note as capture device is not very handy on set.

zcream

I did consider SFF PCs as an option. Some questions came up.

How does one do preview ?

How can I control the camera ?

What about battery options ?

A laptop is the best option as I don't need to worry about a battery or preview. Stereoscopic player allows for a preview.

zcream

Some threads with video capture using open source tools and Flea3. Essentially Flea3 supplies DirectShow dlls This allows compatiblity with VirtualDub. FireCapture also supports video capture - however it seems to be a RAW sequence - not Cinema DNG.

http://www.mathworks.com.au/matlabcentral/newsreader/view_thread/315855

http://www.ptgrey.com/support/kb/index.asp?a=4&q=267

http://www.ptgrey.com/support/kb/index.asp?a=4&q=310&ST=

http://solarchat.natca.net/index.php/en/this-is-solar-chat/6-this-is-solarchat-/83843-for-flea-3-usb3-12q-fl3-u3-13y3m-c-users

zcream

If you can find an old version of Streampix 3 -

http://groups.yahoo.com/neo/groups/Flea3/conversations/topics/429

Here are the steps to activate the Flea3 with StreamPix3:

go to C:\Program Files\Common Files\Norpix\ (assuming that C is the boot drive where all the main programs are installed)
launch the NpxRegMan.exe
click on the Search button and the list of all the available drivers will be created in the window
find the NpxDragonFly entry and check the left hand box
click on the Register button
exit the registration manager and install the Flea3 not forgetting the dongle (you may have to restart the PC)
run StreamPix and select the Point Gray Research choice in the hardware menu
done

zcream

Soeren Mueller was doing a thread on another forum but he disappeared. This is a description of his approach using the pico ITX Via board.

I do not want to use this board as it only has SATA 3Gbps forcing usage of dual SSDs to record 2 streams. My approach is to use SATA 6Gbps and use a fast SSD for writing dual RAW streams.

http://www.viaembedded.com/en/products/boards/1950/1/EPIA-P910.html

I also intend to shut off preview monitoring as the camera will be locked off on a tripod. This should save lots of CPU as there is no need for unnecessary debayering.

Well I'm using MS Dev Studio 2008, writing everything in C++ and have just finished my own DNG writer/export classes that are optimized for speed. I'm using Point Greys SDK to enumerate (more than 1 possible, hello 3D) and control their camera head, set the sensor parameters and read the sensor data via USB3. The data is "preview debayered" for realtime monitoring and can be written to SSD at the same time. So far no real rocket science involved .. building my own touch based interface a la Cinedeck to control everything. Got a small embedded PC board that will run everything and now have to find a nice little highres touch TFT...

Vitaliy_Kiselev

@zcream

You understand that latest Haswell based ITX boards are much better and much more powerful?

zcream

Yup. And possibly fanless as well. The MBP is more because I needed it anyway as a portable editing grading station. And it saves me spending on other equipment.

If I get this running - I will consider a SFF with a touch screen control.

zcream

Freeware MVC video capture programs -

http://www.sharpcap.co.uk/sharpcap

http://firecapture.wonderplanets.de/ - works with flea3

http://noeld.com/programs.asp?cat=video - AmCap used by PGR in their own testing - works with flea3. Trialware costs 12$

http://neuron2.net/LVG/highrescapvdub.html - Virtualdub - if it works, this should be used. Free - regularly updated and has tons of filters.

zcream

Realtime lossless encoding codecs options. Cineform, UTvideo, AMV Codec and x264 lossless

http://stackoverflow.com/questions/6701805/h264-lossless-coding

http://forum.videohelp.com/threads/318382-Lagarith-or-x264-lossless

http://forum.doom9.org/showthread.php?t=158361

http://www.amarectv.com/english/amv2_e.html

https://trac.ffmpeg.org/wiki/x264EncodingGuide

http://forum.doom9.org/showthread.php?t=165745

http://forum.doom9.org/showthread.php?t=165042

http://forum.doom9.org/showthread.php?t=165310

I have used AMV Codec in the past and it works well. x264 lossless is another option.

rrsduncan

What about DPX or JPG2000, they're editable and you're half way to having your film in a showable format or proper DCINE 3D projectors.

zcream

A quad core i7 cannot encode 2x 4K streams of either of these codecs.

Among the codec options -

x264 lossless Hi422P with 10-bit

Cineform 4:2:2 10-bit High

FFV1 4:2:2 10-bit

Both AMV and UTVideo are only 8-bit 4:22

zcream

Hardware and CUDA encoders are also an option as this will allow tasks to be offloaded from the CPU. NOTE :- The problems listed in these threads are not an issue as we are using lossless or low compression x264 encoding with no B-frames.

https://developer.nvidia.com/nvidia-video-codec-sdk

http://www.codeproject.com/Articles/421869/H-264-CUDA-Encoder-DirectShow-Filter-in-Csharp

http://www.tomshardware.com/news/nvidia-gpu-video-encoding-nvenc,21874.html

http://www.behardware.com/articles/828-1/h-264-encoding-cpu-vs-gpu-nvidia-cuda-amd-stream-intel-mediasdk-and-x264.html

http://www.extremetech.com/computing/128681-the-wretched-state-of-gpu-transcoding

NVENC in GTX680 - http://www.guru3d.com/articles-pages/geforce-gtx-680-review,6.html

http://www.extremetech.com/computing/128681-the-wretched-state-of-gpu-transcoding

http://techreport.com/review/23324/a-look-at-hardware-video-transcoding-on-the-pc

http://www.anandtech.com/show/5835/testing-opencl-accelerated-handbrakex264-with-amds-trinity-apu

http://compression.ru/video/codec_comparison/h264_2012/

Unrelated - but shows how fast CoreAVC is - http://forum.doom9.org/showthread.php?t=155441

DGDecNV - http://neuron2.net/dgdecnv/dgdecnv.html

Conclusion :- I can try DGDecNV, CoreAVC and FFMpegSource

zcream

NVEnc is the most promising option ATM

http://arstechnica.com/civis/viewtopic.php?p=23699284&sid=f8468d2429a493c0a426db15b8cfbf63#p23699284

The NVENC test-output was generated by a custom Win32 console-application. I basically combined two NVIDIA SDK samples: "cudaDecode" (which demonstrates how to use the cuVid API to decode MPEG-2/MPEG-4 bitstreams using the NVidia Purevideo dedicated hardware), and nvEncoder (which demonstrates how to compress raw YUV-frames using the NVENC hardware block.) Basically, I specify the source-file (Bluray M2TS file), NVENC encoder parameters/arguments, then the application performs an accelerated decode of the source-video, followed by a accelerated NVENC-encode.

Looks like earlier versions of NVEnc SDK supported consumer CUDA cards and later versions are locked only to Nvidia approved cards.

zcream

Since NvEnc is no longer an option - lets look at Intel Quicksync.

http://babgvant.com/blogs/andyvt/archive/2013/03/18/qstranscode.aspx

From - http://software.intel.com/en-us/articles/harnessing-handbrake

Currently, our experiments using the H.264 Intel® Quick Sync Video codec show Intel HD graphics on Microsoft Windows* 8 utilization approaching 100% while the CPU utilization remains under 20%, allowing for a full speed transcode since the dedicated Intel HD graphics is not affected by the other computing tasks the CPU must always attend to.

http://techreport.com/review/23324/a-look-at-hardware-video-transcoding-on-the-pc/6

For the time being, the best option for quick, high-quality video transcoding is unfortunately to buckle down, get yourself a fast CPU, and run the best software encoder you can find (which may be Handbrake).

From a study at MSU - http://www.compression.ru/video/codec_comparison/h264_2012/

The four best codecs (no codec performs faster with higher quality) in terms of speed/quality are Intel Ivy Bridge QuickSync, MainConcept, Elecard and x264 at average. Absolute speed results are presented in Figure 33 and Figure 34. All the encoders except hardware–based (Intel QuickSync and MainConcept CUDA) have a similar growth rate for encoding time as the bitrate is increased. Intel Ivy Bridge QuickSync is the fastest, followed by DivX H.264.

Haswell includes a hardware MPEG-2 encoder, possibly only 4:2:0 - http://www.anandtech.com/show/6355/intels-haswell-architecture/13

Haswell also adds a hardware motion JPEG decoder, and MPEG2 hardware encoder. With added EUs we'll obviously see QuickSync performance improve, but I don't have data as to how much faster it'll be compared to Ivy Bridge.

qsync - http://tetrachromesoftware.com/index.htm

http://sourceforge.net/projects/qstranscode/

http://www.guru3d.com/files_details/msi_afterburner_beta_download.html

http://www.tomshardware.com/reviews/ivy-bridge-benchmark-core-i7-3770k,3181-7.html

Vitaliy_Kiselev

@zcream

I think that normal codecs comparisons have little value for you.

As you need high speed and high bitrate encoder. While most comparisons are focused on maximum quality using low bitrates.

zcream

True. But I collected the data in case I need to return to it - or if someone finds it useful. Thus far, I think quicksync and quad i7 is the way to go. qsync allows for i-frame only encoding. And I can put the bitrate slider all the way up or quality. Turn off all the other options.

I used this thread to note down all the options - are you concerned that I am posting too much ?

Vitaliy_Kiselev

are you concerned that I am posting too much ?

Of course no.

zcream

Any comments on my requirements so far? You suggested a mini-ITX mobo - I went with a laptop.

Otherwise, it seems to boil down to quicksync and quad i7. I will preview with stereoscopic player. Then stop preview and record. Then playback to preview again. The 480GB SSD with some light compression would easily take 2x 4:2:2 10-bit streams.

In theory, it seems to work.

Vitaliy_Kiselev

I suggest to look for ITX Haswell board with mSATA connector on board. It will be ideal. Very small, low consuming and very powerful.

zcream

Intel QM87, Q87, or H81 ?

Why?

How?

More Details?

Howdy, Stranger!

Categories

Tags in Topic

Top Posters