Personal View site logo
Make sure to join PV on Telegram or Facebook! Perfect to keep up with community on your smartphone.
Please, support PV!
It allows to keep PV going, with more focus towards AI, but keeping be one of the few truly independent places.
Strange Pulsing Diagonal Pattern in GH2 Hack Footage aka rain
  • apefosapefos

    There is a link below in this topic to download an avi file to see the problem, first let me explain it:

    I did a firmware hack only changing the iso limit and set the avchd datarate to 40 Mbps.

    What happens is a strange pulsing diagonal pattern in flat color areas in the image.

    When I play the original mts file with windows media player there is no problem, but...

    When i transcode the original mts file to avi with neoscene the problem is there in the avi file. When i render the original mts file to a h264 file the problem is there in the h264 file.

    Would it be a problem in the hack footage or an issue when decoding avchd?

    Link to download the mp4 video file (10MB) to see the problem:

    http://www.apefos.com/upload/pulsing.mp4

    Any help or knowledge share will be much apreciated. Thanks

  • 70 Replies sorted by
  • apefosapefos

    How I solved/fixed it:

    I perceived that a grey surface is where the rain shows itself more, so I started correct exposure in a flat grey wall and start to underexpose a third fstop each time from f3.5 to f22 to get a dark image.

    the main matrices, (scaling tables) was in a design to be a multiple of 6 or half six in all positions, or it was in a design to be 6 in the dct number and multile of 2 in the other positions. So I did a try to do the same in the deblocking tables, and then the diagonal rain pulsing pattern disappears.

    First I set the deblocking intra and inter with 1 (one) in all positions. This means no quantization at all. Problem was solved but some macroblocks shows up.

    Then I did a try in 6 in all positions, 6 was the dct number in all the scaling tables with other positions multiple of 6 or 3. It worked until the image got 3 fstops underexposed, then the rain appears, so I perceived that I should increase the deblocking values.

    So I increased the deblocking intra all to 36 and keep the deblocking inter all 6, rain disappears. but some mud happens, it was so much deblocking.

    Then I lowered it to 18 in all intraframe positions and to 3 in all the interframe positions. It worked ok, enough debloking and rain keep away. I did a try in 12 in all intra positions but not worked, 18 was ok.

    so the deblocking intraframe is

    018 018 018 018

    018 018 018 018

    018 018 018 018

    018 018 018 018

    (018 in the intra deblocking was ok to deblock the image, no need more, I saw the image frame by frame in 400% zoom, light, shadow and gradients)

    and the deblocking interframe is

    003 003 003 003

    003 003 003 003

    003 003 003 003

    003 003 003 003

    (006 in all positions also works for interframe, but I prefer lower quantization in the interframe, less mud)

    I think the numbers in deblocking tables must be related to the dct number of the main scaling matrices. My scaling matrices has 6 in the dct number so using multiples of 6 or half 6 in the deblocking worked ok. I think if you use 4 in the dct number of the scaling matrices, then the debloking must be multiple of 4 or 2, just a supposition.

    Also I perceived that the main scaling matrices works better when all the numbers have a relationship to be a multiple of some number, example: all multiple of 6, or all multiple of 3, or all multiple of 2. the camera processor does not stress with this design, see the matrices in the "the patch" topic.

  • DrDaveDrDave

    OK I'll bite, how do I fix it? Edit OIC, loading now tum tee tum tee tum....

  • apefosapefos

    Problem is solved.

  • apefosapefos

    The issue is related to the Deblocking Tables.

  • apefosapefos

    It seems I found a way to solve the diagonal rain pulsing pattern problem.

    I am not sure yet.

    I am doing tests.

    If I find it I will upload the patch here:

    http://www.personal-view.com/talks/discussion/11805/the-apefos-settings#Item_45

  • balazerbalazer

    I stumbled across a good example of diagonal rain, and from a few tests it appears pretty clearly to be a problem of the decoder not performing deblocking.

    In-loop deblocking is part of the h.264 spec, and compliant decoders must do it. But some decoders will skip the deblocking to save CPU at the expense of accuracy. Some decoders will skip deblocking only when playback gets behind, so as to allow it a chance to catch up. The errors accumulate over the length of the GOP, so longer GOP footage tends to exhibit the problem more. This is a tricky problem to diagnose: it depends on the encoder, the decoder, and in some cases the conditions at the time of decoding.

    I've uploaded three files: http://www.mediafire.com/?aqw7u5x2g2x8u

    • a raw camera GH2 MTS file that shows diagonal rain when deblocking is disabled

    • a file transcoded with deblocking disabled in the decoding (shows rain)

    • a file transcoded with deblocking enabled in the decoding (no rain)

    Both transcoded files were encoded with deblocking disabled.

    I slowed and lightened this video for YouTube to make the rain clearer through YouTube's compression. Make sure you switch the quality to 720p.

    Using ffdshow as your decoder, there are three settings that must be unchecked to ensure deblocking is always performed: in Codecs, with h.264/AVC selected, uncheck "skip deblocking when safe” and “skip deblocking always”. In Decoder Options, uncheck “skip h264 deblocking on delay”.

  • balazerbalazer

    Disabling deblocking saves CPU time. It trades accuracy for speed, when your CPU might otherwise be too slow to play back the video.

    Deblocking is part of the h.264 spec. Compliant decoders all do deblocking the same way.

  • SilvercatSilvercat

    @LPowell I'm using from many months this hack on my cineform conversion. Quality is very good and no rain effect. And, on the other hand, some h264 decoder (like coreavc) have a better deblocking, o different from mainconcept. Coreavc with deblocking activate don't have any rain artifacts. I know you are right, but the fact is that the rain effect is worse than decode without deblocking.

    A question: If the deblocking filter is so important and needed, why some decoder have the option for disable it? In Coreavc you can disable deblocking in GUI, and also some mainconcept installation.

  • LPowellLPowell

    @radikalfilm

    ...why do you have to be so smart and yet put out a "broken" patch? just saying.

    Go ahead, blame the messenger all you want ;-P

  • radikalfilmradikalfilm

    @lpowell blimey now I have to UNDO those reg changes...why do you have to be so smart and yet put out a "broken" patch? just saying. Not personal.

  • LPowellLPowell

    @willyfan

    This IS a deblocking decode filter problem, absolutely. And it is a problem from mainconcept. The best thing is disable deblocking in ALL mainconcept filter installed.

    This hack would be worse than the digital rain disease it's intend to cure. The decoder's deblocking filter is a crucial component needed to smooth the horizontal and vertical edges between H.264 macroblocks. The encoder uses its deblocking filter to construct the internal reference frames that are used as the starting point to encode P and B-frames. If the decoder does not use its deblocking filter to reconstruct the reference frames in exactly the same way the encoder did, decoded P and B-frames will be contaminated with macroblock artifacts that cannot be cleanly filtered out in post.

  • balazerbalazer

    I want to test different decoders.

  • radikalfilmradikalfilm

    @balazer I could. You want to see what it looks like?

  • balazerbalazer

    Can anyone provide a raw MTS file that has rain when decoded with the bad decoders?

  • SilvercatSilvercat

    @apeflos No advantage with mainconcept GUI. Avery software use his own filter, so you change in GUI only the filter installed in directshow but not the cineform one, nor CS6. The value 1, "reference only" I think that it apply deblock only on I frame. I don't know if it is a advantage, I think that is better "disable" @meierhans I tried to work on Adobe importer. Adobe use for h264 import a plugin named "ImporterMPEG.prm" located in polugin folder. May be that with a patch on this file it work... Don't try to change this file with the CS 5.5 one: don't work!! Unfortunately, the only alternative for the import plugin h264 in CS6 is always MainConcept, I tried it but it has the same problem and you can not disable the deblocking. @nomad I don't have a MAC for test, If someone want try, I suggest to start from System Preferences > Choose View > QuickTime > Advanced And check if there is the deblocking option.

  • nomadnomad

    Did anybody find a way to disable it on the Mac?

    TIA

  • apefosapefos

    @willyfan thanks for the new information. did you find difference when using 1 or 2? the value 1 also works to avoid the GH2 problem or just the value 2 works?

    Also, are there advantages in installing the Mainconcept GUI? Changing values in the GUI can avoid changing values in windows register? Is it a software for transcoding video or a codec pack?

  • MeierhansMeierhans

    Wish we could find a way to disable it for CS6! Maybe if you patch some files?

  • SilvercatSilvercat

    Allowed value for "deblocking" are 0, 1 or 2. 0 mean "standard" 1 mean "reference only" (don't ask to me what is this...) 2 mean "disabled"

  • SilvercatSilvercat

    I have a mainconcept version with GUI set. This filter is not used from cinestar nor Adobe, but I was able to discover the difference in register from deblocking ON and OFF. I used this filter with graphedit for test, so I'm sure that this is the problem. Default is zero, with deblocking ON. If I set deblocking OFF the value change to 2. My test on the cineform mainconcept filter work well with 2 value. Jast I have more time, I can try to understand what happen with value set to 1.

  • radikalfilmradikalfilm

    @willyfan Yes do tell us why 2 and not 1 or 3. Is it a dial, and if so, what does it dial exactly?

  • apefosapefos

    @willyfan thanks for this. Today the 48Mbps GOP3 is working good for me, but if some day I get into trouble again I will try your approach.

    Can you explain why change from zero to 2? Does it have more options like 1 or 3? And what does these numbers means?

  • SilvercatSilvercat

    This IS a deblocking decode filter problem, absolutely. And it is a problem from mainconcept. Cineform converter use a own mainconcept filter, but is possible to disable deblocking in register. The best thing is disable deblocking in ALL mainconcept filter installed.

    In HKEY_CURRENT_USER / Software / must find all folders named "MainConcept". Enter all those concerning the decompression h264 (eg "MainConcept AVC/H.264 Video Decoder) and in "default" folder set the deblocking value to 2 instead of zero. If there is a folder HDLink.exe , do the same in that. HDLInk.exe is the program for conversions CineForm, who was using another might find other names. it 'a good thing to open all folders and search for the item Deblocking and take it to 2.

    Unfortunately, I've found no way to disable the deblocking filter to import cs6 (may be a 64 bit issue)

  • DrDaveDrDave

    So, if I upload five clips, unlabelled, three of which are identical, and three of which use different patches, you could tell me which ones have certain aesthetic qualities, and also do that without accidentally assigning different aesthetic qualities to identical patches--because three of them, unlabelled, would be the same?

    As for disk "cheap, fast and plentiful", if you go through 2TB a month, minimum, and can go 6TB in a busy time, and you back up your drives which I have to do because, ummm, the alternative is bleak, that's 4TB to 12TB.

    If you don't shoot a lot of video, then you are right, a few big drives will suffice. If you do shoot a lot of video, then it is a problem. And, here, the problem is centered around a current trend in using lots of cameras at high bitrates. Why use one camera when you can use ten?

    The money you save is more than enough to buy a new camera and a nice lens every year. New cameras and lenses give you a competitive advantage in the marketplace. Or, you could use your newly minted gold coins to purchase a matched pair of Schoeps MK 41s, which will make all your soundtracks sound amazing.

    In fact, the money you save, if invested in a real top end lens, will actually create more of a difference in the quality of the video than the difference among the various patches and resampling techniques.

    Ironic, isn't it? (It's OK nowadays to substitute irony for sarcasm, the rules were loosened).

    Right now I have two walls completely full of hard drives, and cutting that by 75 percent is in fact a real issue. I don't live in a mansion, but even if I did I wouldn't fill it with hard drives. Anyone who want to fill their house with hard drives, that's OK with me.

    If there is a breakthrough in storage, and 20TB was freely available on a USB stick, I would reconsider it.

    But there is another issue, which is you are resampling your video. Some people don't want to do that.

    And, I hesitate to mention this, but you are misusing the word ironic. Which is ironic, when you think about it.

  • BurnetRhoadesBurnetRhoades

    Disk is cheap, fast and plentiful. You'd rather chase non-solutions and waste time and effort trying to be frugal. Spend time and effort in the pursuit of doing less work. Y'all are absolutely ironic.

    "Do what you like." --Tyler Durden

Add a Comment