Personal View site logo
Make sure to join PV on Telegram or Facebook! Perfect to keep up with community on your smartphone.
Please, support PV!
It allows to keep PV going, with more focus towards AI, but keeping be one of the few truly independent places.
Pro: AVCHD Quantization process
  • As we are moving deeper I am making this topic to talk about some AVCHD encoder things.

    It is about QStep table (52 elements table each [i+6] item is doubled compared to [i] item).
    As I am looking at this table in GH2 video encoder code.
    I want to complety understand its role in quanization.

    Useful links:
    http://www.vcodex.com/files/h264_transform.pdf
    http://www.vcodex.com/files/H264_4x4_transform_whitepaper_Apr09.pdf
    http://www.addpac.com/addpac_eng2/down.php?file=267_f11.pdf
    http://www.amazon.com/MPEG-Handbook-Second-John-Watkinson/dp/024080578X/ref=sr_1_1?ie=UTF8&qid=1310313414&sr=8-1
    http://www.amazon.com/H-264-Advanced-Video-Compression-Standard/dp/0470516925/ref=sr_1_1?ie=UTF8&qid=1310313609&sr=8-1
    http://www.amazon.com/H-264-MPEG-4-Video-Compression-Generation/dp/0470848375/ref=sr_1_2?ie=UTF8&qid=1310313609&sr=8-2

    Do not post there if you do not understand that it is about.
    Any stupid questions will be deleted instantly.
    Otherwise any info from knowledgeable people are welcome.
  • 131 Replies sorted by
  • So what is way of signals from sensor to final image? Wich step we can handle?

  • @cbrandin Thank you for the concrete and detailed explanation.

    "This is what happened in the examples you posted where the min QP is 9 and the max is 35."

    I was afflicted by this phenomenon all the time. "Whether this is good or bad," I could not be determined... Many thanks to the clear understanding that you do. Use the proper frame size and Q, and I think let's make the correct patch. Thanks again. :-)

    BTW, I always use the Stray's chart has been introduced by qwerty123.

  • @driftwood I understand thinking that you are interesting. Of course, although, as for the result of a test chart, a buffer falls to zero in several seconds, recording does not stop. However, since I have set up so that a buffer may not fall in the practical use range, there is no problem in actual shooting, and since recording does not stop even if a buffer should fall, I thought that it is satisfactory. Although the affair of image quality is another question again ... I think that image quality is very bad after frame size becomes small. However again, there is no problem in actual shooting...

    @Vitaliy_Kiselev Although I thought that the thread here was suitable, please move, supposing there is another suitable thread.

    Thank you always. :-)

    q13_143m_theory_5s.JPG
    676 x 823 - 64K
  • @qwerty123

    Thanks - that's the one...

  • @cbrandin Thanks for that detailed explanation. I'm still a little hazy about the Top and Bottom settings. Could you explain how they work in the scheme of things?

    Thanks.

  • By the way, when testing with charts you should use one of the color "crash test" charts, not a monochrome one. The monochrome chart contains no color data, so it won't stress the codec as much. I know somebody produced a color version of the popular test chart used around here.

    Chris

  • The AQ setting really only works correctly with normal GOP settings. With short GOP you have to set Q manually to get the right results.

    There are four settings that need to be balanced: Q, Frame Limit, Video Bitrate, and GOP.

    Q and Frame Limit are set depending on how you set Video Bitrate and GOP. If you compare GOP configurations you have to consider that low GOP settings need to have higher Video Bitrates to support the same Q value. For example:

    The default Q value is 20 for 1080p at 22M at GOP12. To support the same Q value with GOP6, you need about 40M, with GOP3 you need around 80M, with GOP1 you need about 190M. That means that if you are running GOP3 at 143M, you would need to set Q to around 15-16. If you were using GOP1, you would actually need to set Q to about 22-23. Or, put another way, GOP6 requires approximately 1.8x the bandwidth of GOP12, GOP 3 approximately 3.6x, and GOP1 approximately 8.6x.

    The default Frame Limit is 7864320. The highest frame limit that actually does anything is about 10000000. You can't raise Frame Limit unless the Q has been properly reduced, and you must lower it if Q has been set to over 20. In the above GOP3 example you could raise it to 10000000. In the case of GOP1 at 143M you would have to lower the Frame Limit to 5958333. The GOP1 calculation is simple - take the Video Bitrate and divide it by # frames per second: 143/24=5958333. There is another parameter that affects all this - xxxx Top Setting (xxx is the mode you are setting). Your actual Video Bitrate is the lower of the Video Bitrate setting or the Top setting*1000. If you haven't checked the Top setting you can just use the Video Bitrate setting.

    These settings only have the desired effect if they are within the bounds of what the codec can handle. Bad things happen when these are not calculated correctly. For example, if you set Q too low you end up with frames that have some macroblocks coded at the desired Q value, but because the codec runs out of bandwidth it will encode other macroblocks at a much higher value in order to avoid crashing - or, some macroblocks, or even entire frames will be skipped. This is what happened in the examples you posted where the min QP is 9 and the max is 35. This should not happen, especially with a chart. If the difference between min and max values is more than around 2 with charts then you have a problem. The same kind of thing can happen if you set Frame Limit too high - but in that case it will be the P and B frames that fail most often, or the codec will crash altogether.

    Basically, you want to set these parameters so that test charts result in well behaved streams. With the "Room" clips you would expect bitrates to drop, or QP to come out lower. That's how VBR codecs work - when there is less detail the bitrate goes down. Having the codec work right with low detail scenes only is meaningless.

    Chris

  • Are you sure this is proper topic for this patches testing?

    I want to keep this topic tech only discussion.

  • @bkmcwd Hmm... an intersting comparison, but I find the encoder begins to settle more 5 secs in, could you upload some Q analysis at least 10 seconds into the recording? Many thanks for these results.

  • @driftwood These are old and new comparison of AQ2 and Q13. In each, the upper row is in my room and the lower berth is a result of a Stray's chart.

    In Q13, I think that there is no clear difference. However, there was a clear difference in AQ2. The minimum qp and the highest qp have shifted in old and new. Do you think that which is good as for image quality?

    Thanks for your advice! :-)

    NOTE: The result was the same although various encoder settings were also changed. Then, furthermore, as a result of raising accuracy, both were set to 24H=143M and 24L=117.9M also AQ2 and Q13.

    AQ2.jpg
    1349 x 1691 - 444K
    Q13.jpg
    1349 x 1692 - 456K
  • @driftwood Thanks for always helping me, mate. :-)

    "You need a balance of resolution and good Q."

    What kind of thing is this concretely?

    I am confused. :-(

    Since I think that you can also understand my strange English, I ask you. It is recommended by you and I came to look at macroblocks. Although I understood that low qp produced high definition, if I try to use low qp like the upper sample, high qp will be mixed because of the limit of frame size. On the other hand, although there is no extremely low qp when it sets to a certain value of low qp, high qp also stops mixing, like the upper sample. Although I do not understand controlling qp how for whether it is best, I think that big frame size is required to lower the average value of the whole screen at least. Naturally qp low on the average enlarges frame size.

    Many of my friends think that higher resolving can be obtained with big frame size. Although I can understand their idea, I cannot do proving it well.

    If high definition can be obtained regardless of the size of frame size, I think that all the GH2 users SHOULD use your patch silently. It is because I understand well that your patch is rational and wonderful.

    For example, if there is a conclusion "AQ4 is high definition under the Lowlight, and resolving becomes low from AQ2 under the usual brightness", I would like to know it. If there is such a conclusion, as for 3GOP which can realize big frame size so that my friends may think, coexisting with rational Intra has a meaning.

    Thanks for your advice!

  • You need a balance of resolution and good Q.

  • @cbrandin Thanks for your great work. :-)

    I have a question to you. These are the results of shooting with the tea garden using my Q13, AQ4, and Q18 all 3GOP patches. The first screen shot is a result in each Streamparser. The second is a result of StreamEye. The third is the result of my investigating 40 macroblocks from 40x20 to 43x29 of the screen of Streameye.

    Is an image quality of which the highest among these three patches results?

    Although I thought that AQ4 was always high definition, in about 800k small frame size, qp will become high on the average because of extremely low qp. Then, I thought that the patch of the big frame size set to qp low rather on the average became high definition more. How do you think?

    ISO160_StreamParser.jpg
    1296 x 1898 - 889K
    ISO160_StreamEye.jpg
    2027 x 825 - 287K
    Q13_AQ4_Q18.JPG
    472 x 839 - 119K
  • @driftwood Thanks mate! Since I was looking for this thread, I was saved. :-) And your test result is very interesting and helps me.

    I think that it is utilizing the big frame size as a merit of 3GOP, and being able to apply qp low on the average to the whole screen these days. If the average value in the whole screen of qp is considered as I reported in the LOW GOP thread before, I think that qp low on the average will be applied for AQ2 with larger frame size than AQ4 with small frame size.

    Furthermore, I also think that there will also be a case which leads to the improvement in image quality with more nearly overall setting up mean q value by a manual in fact rather than applying very low min qp by AQ4. I think that the size of frame size is closely related, of course also in this case.

  • The same test on a faster lens - f2 Olly or voigt 0.95, requiring a much lower ISO (160) at 10iq/10q shows zero quant lines at all - ie everything is Q'd nicely (which suggest AQ4 can be used fine with decent lenses/lower ISO). This suggests once again that low iso or fast lens combination is imperative for the GH2. I am also unimpressed with the Panny lenses which come with camera. Bring on some decent fast lenses from them.

  • Test 3) See graphic below. AQ switched off or Stock factory iQ/Q (20) Graphic. Notice: The huge drop in bitrate since very little use of the bitrate setting is now being utilised by this poorer IQ/Q and 5% worse EPSNR (estimated signal noise ratio).

    Test 4) *No graphic shown. If you try IQ=10, Q=20 manual will give you an opening frame of minQ16(maxQ16) and avg mbs QP on all other frames of between 18-22 (almost as Test3 but with the better initial frame) plus the huge drop in bitrate resource as Test 3.

    Test 5) *No Graphic shown. If you try iQ=20, Q=10 manual, will give you an opening frame of (Q20min/Q20max), then an avg mbs QP in all other frames of between 8 and 14 as almost comparable to the (IQ10/Q10) setting seen in test 1.

    AQ0 (or AQ switched off).png
    963 x 1041 - 635K
  • Here is a visual test showing hi bitrate INTRA 1080p with settings iQ=10, Q=10 with AQ switched off (Test1, see left pane view) versus the AQ4 setting with GH2's default IQ=20, Q=20 (Test2, see right hand pane). Notice the lined quantisation difference with the manual setting (barely any) against the AQ4 'very lined' (alternating lines mean light lines better Q and dark lines worse Q). The first graphic is the picture view at frame 0 on both streams. The second graphic is a visual of the Quantisers both streams at frame 45. The third graphic is a mb sizes view showing macroblocking at frame 1 on both streams. The test was conducted from a moving recording on a motorised tripod. The noise is due to 1600ISO, 50 shutter in a lit room, on a 14-140 panny lens (set at 18mm).

    Findings: Notice how by setting it to 10 & 10 in the manual recording the actual IQ and Q now = 16!!! The lower IQ & Q has also given a more defined image in resolution, see the woman's back, however, AQ4 blurs these more if you look hard. The AQ4 on this hi bitrate gives a alternating low/high lined quantisation, whereas the manual IQ/Q doesnt so much.

    The frame min QP and Max QP average out as follows. Setting IQ=10,Q=10 produced an avg 'frame Qp' of between 8min and 15max Setting it to AQ4 only produced an avg 'frame QP' of between 2min and 18max However, the avg macroblock Qp was almost constant 12 for the IQ/Q=10 setting and, the avg macroblock Qp for the AQ4 test was alternating line by line from 10 to 15 - very weird.

    Discuss.

    AQ4-iq20-Q20 versus manual iq10q10 - Frame 0 - Picture Visual view.png
    1920 x 1080 - 2M
    AQ4-iq20-Q20 versus manual iq10q10 - Quantisers Visual view.png
    1920 x 1080 - 151K
    AQ4-iq20-Q20 versus manual iq10q10 - Frame 1 mb sizes view.png
    1920 x 1080 - 211K
  • Here is some grabs from Elecard Streameye showing quantisers/mb sizes on a picture.
    Here's a great link about Rate Control and h264
    http://www.pixeltools.com/rate_control_paper.html
    elecard mb types lines.png
    1827 x 882 - 898K
    elecard mb size lines.png
    1826 x 865 - 219K
    elecard quantiser lines.png
    1809 x 868 - 183K
  • @driftwood

    Sure, this is after all a public forum.

    StreamEye does correctly identify that there are no 8x8 blocks in the video data, it's just the showing of the tables in the headers that is problematic - and that may not be their bug. For example, the stream may report that the tables are there, not provide them and not use them - I don't know, I haven't looked that deep into it.

    There are a couple of other things Elecard might consider adding. StreamEyE doesn't provide "MTS" files as a compatible option, to load the files you have to select "All" file types. Also, the tips (pop-ups) feature is quirky, you have to select the display window by clicking on the title bar to make them work, clicking inside the window doesn't seem to work.

    By the way, have you noticed their "Slim" version of StreamEyE for $50? I doesn't have all the features of regular StreamEye, but I have found it very useful nevertheless - and the price is right.

    Chris
  • @cbrandin Elecard are looking into a bug their end. Thanks so much for proving this. Is it ok if I direct them to here?
  • Here's the active picture parameter set for an I frame as reported by CodecVisa - the 8x8 matrix flag is 0 (no 8x8 matrices):

    image
  • Here's a screenshot from CodecVisa (another analysis program) - it shows the entire layout for an I frame macroblock. As you can see, it's made up entirely of 4x4 blocks:

    image
  • The H.264 standard also describes Arbitrary Slice Ordering (ASO), Flexible Macroblock Ordering (FMO), Scalable Video Coding (SVC), etc, etc... none of which is used in the GH2. I think you're really confusing the issue here.

    Chris
  • @driftwood

    Those screengrabs have nothing to do with the GH2. If you read further in his book he stipulates that 8x8 blocks are an option - not a requirement. Also if you read the part where he explains how quantization scaling matrices work, you'll understand that the values in those 8x8 matrices showing up in the GH2 dumps are impossible. Also, I've studied the firmware tables pretty deeply - there are no 8x8 matrices, and I have actually seen all the 4x4 matrices that show up in the streams in the firmware tables.

    If you look at any of the frames using StreamEye, and examine the individual macroblocks, you'll see that none of them use 8x8 blocks, they all use 4x4 blocks:



    image