Personal View site logo
Make sure to join PV on Telegram or Facebook! Perfect to keep up with community on your smartphone.
Pro: AVCHD Quantization process
  • 131 Replies sorted by
  • @cbrandin Have you taken a look at elecard stream analyser yet?
  • @driftwood
    he know about this for a year at least (I think longer :-) )
  • @cbrandin The reason I ask is because sometimes if you close and then open Stream Analyser and reopen the same mts file sometimes you can get conflicting results in the matrices - especially the 8x8s. See my reopen file tests. These are all conducted on the first slice at memory address 0x000003AF .

    Edit: Summary - The 8x8 matrices are changing flags & matrix patterns. e.g.
    *** 1st open of program and file
    pic_scaling_list_present_flag[6] = 63
    use_default_scaling_matrix_flag8x8[6] = 0
    pic_scaling_list_present_flag[7] = 86
    use_default_scaling_matrix_flag8x8[7] = 0


    *** 2nd open of program and file
    pic_scaling_list_present_flag[6] = 63
    use_default_scaling_matrix_flag8x8[6] = 1
    pic_scaling_list_present_flag[7] = 243
    use_default_scaling_matrix_flag8x8[7] = 0
    and so on - theyre all different!

    Therefore could you check this is whats happening to you too?
    test to see stream analsyer is stable with a gh2 file.txt
  • @driftwood

    I'm very familiar with StreamEye. The 8x8 matrices don't really exist - StreamEye is falsely interpreting other data that follows as 8x8 matrices. I don't know if this is a bug in StreamEye or some flag being set incorrectly by the GH2. One thing I do know is that the 8x8 matrix values shown in StreamEye are random garbage. It's perfectly acceptable to not use any 8x8 matrices under the standard.

    The first GOP values are all from the T4 matrices - the "I give up" matrices. This is probably caused by a bug we are well familiar with that is being fixed in the next release of PTool.

  • @cbrandin Yep, guess youre right. Its just garbage. I'm giving up on these time consuming matrix tests. Shame elecard stream analyser doesnt appear to work correctly (maybe it will eventually) cus it would be great to analyse the gh2 streams and actually see whats going on. :-( I eagerly await the next ptools update. thanks for the info.
    PS tried the Tektronix MTS4EA stream analyser but it doesnt allow external file import in the demo version :-(
  • @cbrandin Actually I'm beginning to think those two 8x8 matrices are being used as frequency dependent quantisation. The value 'seq_scaling_matrix_present_flag = 1' in the Sequence Parameter Set and direct_8x8_inference_flag = 1 (which follows further down) seems to confirm this. If you look at The H.264 Advanced Video Compression Standard By Iain Richardson he concludes in the following three screengrabs...>

    EDIT: For anyone thinking of further background reference there is downloadable AVC/h.264 standard revisions at:
    defualt quantization and frequency dependent quantization1.png
    443 x 310 - 23K
    defualt quantization and frequency dependent quantization2.png
    561 x 604 - 46K
    defualt quantization and frequency dependent quantization3.png
    558 x 526 - 36K
  • @driftwood

    Those screengrabs have nothing to do with the GH2. If you read further in his book he stipulates that 8x8 blocks are an option - not a requirement. Also if you read the part where he explains how quantization scaling matrices work, you'll understand that the values in those 8x8 matrices showing up in the GH2 dumps are impossible. Also, I've studied the firmware tables pretty deeply - there are no 8x8 matrices, and I have actually seen all the 4x4 matrices that show up in the streams in the firmware tables.

    If you look at any of the frames using StreamEye, and examine the individual macroblocks, you'll see that none of them use 8x8 blocks, they all use 4x4 blocks:

  • The H.264 standard also describes Arbitrary Slice Ordering (ASO), Flexible Macroblock Ordering (FMO), Scalable Video Coding (SVC), etc, etc... none of which is used in the GH2. I think you're really confusing the issue here.

  • Here's a screenshot from CodecVisa (another analysis program) - it shows the entire layout for an I frame macroblock. As you can see, it's made up entirely of 4x4 blocks:

  • Here's the active picture parameter set for an I frame as reported by CodecVisa - the 8x8 matrix flag is 0 (no 8x8 matrices):

  • @cbrandin Elecard are looking into a bug their end. Thanks so much for proving this. Is it ok if I direct them to here?
  • @driftwood

    Sure, this is after all a public forum.

    StreamEye does correctly identify that there are no 8x8 blocks in the video data, it's just the showing of the tables in the headers that is problematic - and that may not be their bug. For example, the stream may report that the tables are there, not provide them and not use them - I don't know, I haven't looked that deep into it.

    There are a couple of other things Elecard might consider adding. StreamEyE doesn't provide "MTS" files as a compatible option, to load the files you have to select "All" file types. Also, the tips (pop-ups) feature is quirky, you have to select the display window by clicking on the title bar to make them work, clicking inside the window doesn't seem to work.

    By the way, have you noticed their "Slim" version of StreamEyE for $50? I doesn't have all the features of regular StreamEye, but I have found it very useful nevertheless - and the price is right.

  • Here is some grabs from Elecard Streameye showing quantisers/mb sizes on a picture.
    Here's a great link about Rate Control and h264
    elecard mb types lines.png
    1827 x 882 - 898K
    elecard mb size lines.png
    1826 x 865 - 219K
    elecard quantiser lines.png
    1809 x 868 - 183K
  • Here is a visual test showing hi bitrate INTRA 1080p with settings iQ=10, Q=10 with AQ switched off (Test1, see left pane view) versus the AQ4 setting with GH2's default IQ=20, Q=20 (Test2, see right hand pane). Notice the lined quantisation difference with the manual setting (barely any) against the AQ4 'very lined' (alternating lines mean light lines better Q and dark lines worse Q). The first graphic is the picture view at frame 0 on both streams. The second graphic is a visual of the Quantisers both streams at frame 45. The third graphic is a mb sizes view showing macroblocking at frame 1 on both streams. The test was conducted from a moving recording on a motorised tripod. The noise is due to 1600ISO, 50 shutter in a lit room, on a 14-140 panny lens (set at 18mm).

    Findings: Notice how by setting it to 10 & 10 in the manual recording the actual IQ and Q now = 16!!! The lower IQ & Q has also given a more defined image in resolution, see the woman's back, however, AQ4 blurs these more if you look hard. The AQ4 on this hi bitrate gives a alternating low/high lined quantisation, whereas the manual IQ/Q doesnt so much.

    The frame min QP and Max QP average out as follows. Setting IQ=10,Q=10 produced an avg 'frame Qp' of between 8min and 15max Setting it to AQ4 only produced an avg 'frame QP' of between 2min and 18max However, the avg macroblock Qp was almost constant 12 for the IQ/Q=10 setting and, the avg macroblock Qp for the AQ4 test was alternating line by line from 10 to 15 - very weird.


    AQ4-iq20-Q20 versus manual iq10q10 - Frame 0 - Picture Visual view.png
    1920 x 1080 - 2M
    AQ4-iq20-Q20 versus manual iq10q10 - Quantisers Visual view.png
    1920 x 1080 - 151K
    AQ4-iq20-Q20 versus manual iq10q10 - Frame 1 mb sizes view.png
    1920 x 1080 - 211K
  • Test 3) See graphic below. AQ switched off or Stock factory iQ/Q (20) Graphic. Notice: The huge drop in bitrate since very little use of the bitrate setting is now being utilised by this poorer IQ/Q and 5% worse EPSNR (estimated signal noise ratio).

    Test 4) *No graphic shown. If you try IQ=10, Q=20 manual will give you an opening frame of minQ16(maxQ16) and avg mbs QP on all other frames of between 18-22 (almost as Test3 but with the better initial frame) plus the huge drop in bitrate resource as Test 3.

    Test 5) *No Graphic shown. If you try iQ=20, Q=10 manual, will give you an opening frame of (Q20min/Q20max), then an avg mbs QP in all other frames of between 8 and 14 as almost comparable to the (IQ10/Q10) setting seen in test 1.

    AQ0 (or AQ switched off).png
    963 x 1041 - 635K
  • The same test on a faster lens - f2 Olly or voigt 0.95, requiring a much lower ISO (160) at 10iq/10q shows zero quant lines at all - ie everything is Q'd nicely (which suggest AQ4 can be used fine with decent lenses/lower ISO). This suggests once again that low iso or fast lens combination is imperative for the GH2. I am also unimpressed with the Panny lenses which come with camera. Bring on some decent fast lenses from them.

  • @driftwood Thanks mate! Since I was looking for this thread, I was saved. :-) And your test result is very interesting and helps me.

    I think that it is utilizing the big frame size as a merit of 3GOP, and being able to apply qp low on the average to the whole screen these days. If the average value in the whole screen of qp is considered as I reported in the LOW GOP thread before, I think that qp low on the average will be applied for AQ2 with larger frame size than AQ4 with small frame size.

    Furthermore, I also think that there will also be a case which leads to the improvement in image quality with more nearly overall setting up mean q value by a manual in fact rather than applying very low min qp by AQ4. I think that the size of frame size is closely related, of course also in this case.

  • @cbrandin Thanks for your great work. :-)

    I have a question to you. These are the results of shooting with the tea garden using my Q13, AQ4, and Q18 all 3GOP patches. The first screen shot is a result in each Streamparser. The second is a result of StreamEye. The third is the result of my investigating 40 macroblocks from 40x20 to 43x29 of the screen of Streameye.

    Is an image quality of which the highest among these three patches results?

    Although I thought that AQ4 was always high definition, in about 800k small frame size, qp will become high on the average because of extremely low qp. Then, I thought that the patch of the big frame size set to qp low rather on the average became high definition more. How do you think?

    1296 x 1898 - 889K
    2027 x 825 - 287K
    472 x 839 - 119K
  • You need a balance of resolution and good Q.

  • @driftwood Thanks for always helping me, mate. :-)

    "You need a balance of resolution and good Q."

    What kind of thing is this concretely?

    I am confused. :-(

    Since I think that you can also understand my strange English, I ask you. It is recommended by you and I came to look at macroblocks. Although I understood that low qp produced high definition, if I try to use low qp like the upper sample, high qp will be mixed because of the limit of frame size. On the other hand, although there is no extremely low qp when it sets to a certain value of low qp, high qp also stops mixing, like the upper sample. Although I do not understand controlling qp how for whether it is best, I think that big frame size is required to lower the average value of the whole screen at least. Naturally qp low on the average enlarges frame size.

    Many of my friends think that higher resolving can be obtained with big frame size. Although I can understand their idea, I cannot do proving it well.

    If high definition can be obtained regardless of the size of frame size, I think that all the GH2 users SHOULD use your patch silently. It is because I understand well that your patch is rational and wonderful.

    For example, if there is a conclusion "AQ4 is high definition under the Lowlight, and resolving becomes low from AQ2 under the usual brightness", I would like to know it. If there is such a conclusion, as for 3GOP which can realize big frame size so that my friends may think, coexisting with rational Intra has a meaning.

    Thanks for your advice!

  • @driftwood These are old and new comparison of AQ2 and Q13. In each, the upper row is in my room and the lower berth is a result of a Stray's chart.

    In Q13, I think that there is no clear difference. However, there was a clear difference in AQ2. The minimum qp and the highest qp have shifted in old and new. Do you think that which is good as for image quality?

    Thanks for your advice! :-)

    NOTE: The result was the same although various encoder settings were also changed. Then, furthermore, as a result of raising accuracy, both were set to 24H=143M and 24L=117.9M also AQ2 and Q13.

    1349 x 1691 - 444K
    1349 x 1692 - 456K
  • @bkmcwd Hmm... an intersting comparison, but I find the encoder begins to settle more 5 secs in, could you upload some Q analysis at least 10 seconds into the recording? Many thanks for these results.

  • Are you sure this is proper topic for this patches testing?

    I want to keep this topic tech only discussion.

  • The AQ setting really only works correctly with normal GOP settings. With short GOP you have to set Q manually to get the right results.

    There are four settings that need to be balanced: Q, Frame Limit, Video Bitrate, and GOP.

    Q and Frame Limit are set depending on how you set Video Bitrate and GOP. If you compare GOP configurations you have to consider that low GOP settings need to have higher Video Bitrates to support the same Q value. For example:

    The default Q value is 20 for 1080p at 22M at GOP12. To support the same Q value with GOP6, you need about 40M, with GOP3 you need around 80M, with GOP1 you need about 190M. That means that if you are running GOP3 at 143M, you would need to set Q to around 15-16. If you were using GOP1, you would actually need to set Q to about 22-23. Or, put another way, GOP6 requires approximately 1.8x the bandwidth of GOP12, GOP 3 approximately 3.6x, and GOP1 approximately 8.6x.

    The default Frame Limit is 7864320. The highest frame limit that actually does anything is about 10000000. You can't raise Frame Limit unless the Q has been properly reduced, and you must lower it if Q has been set to over 20. In the above GOP3 example you could raise it to 10000000. In the case of GOP1 at 143M you would have to lower the Frame Limit to 5958333. The GOP1 calculation is simple - take the Video Bitrate and divide it by # frames per second: 143/24=5958333. There is another parameter that affects all this - xxxx Top Setting (xxx is the mode you are setting). Your actual Video Bitrate is the lower of the Video Bitrate setting or the Top setting*1000. If you haven't checked the Top setting you can just use the Video Bitrate setting.

    These settings only have the desired effect if they are within the bounds of what the codec can handle. Bad things happen when these are not calculated correctly. For example, if you set Q too low you end up with frames that have some macroblocks coded at the desired Q value, but because the codec runs out of bandwidth it will encode other macroblocks at a much higher value in order to avoid crashing - or, some macroblocks, or even entire frames will be skipped. This is what happened in the examples you posted where the min QP is 9 and the max is 35. This should not happen, especially with a chart. If the difference between min and max values is more than around 2 with charts then you have a problem. The same kind of thing can happen if you set Frame Limit too high - but in that case it will be the P and B frames that fail most often, or the codec will crash altogether.

    Basically, you want to set these parameters so that test charts result in well behaved streams. With the "Room" clips you would expect bitrates to drop, or QP to come out lower. That's how VBR codecs work - when there is less detail the bitrate goes down. Having the codec work right with low detail scenes only is meaningless.


  • By the way, when testing with charts you should use one of the color "crash test" charts, not a monochrome one. The monochrome chart contains no color data, so it won't stress the codec as much. I know somebody produced a color version of the popular test chart used around here.