<< ctxmodel.net
Here's the BWT output CM compression comparison, version 2.
o1rc9f added:
- rc_sh2f bitwise 32-bit arithmetic coder added, but didn't change anything,
so the planned implementation of 64-bit coder was cancelled;
- Node2i update tables are now generated at runtime, and static tables which were
used before had been removed... still this allowed to win only 50 bytes on calgary;
- mixer limit masks were expanded (another -100 bytes);
- some minor Node2i update modifications and a lot of various reformatting which didn't affect the compression.
o1rc9g added:
- adding run-lengths to mixer contexts failed (adding it to counter contexts failed earlier);
- using the o1rc9f for ranking and additional order0 for coding (secondary model)...
not exactly failed, but its results were too pitiful to try adding it to the mix or something;
- another order0 submodel was added and mixed with other two order0's before
the mix with order1, but then it was turned into order1 by optimizer, and after
tuning it shows better results than the order2 coder... Most funny thing though, is that
this new order1 submodel is so "fast", that it practically assigns all the probability to
the LES (last encountered symbol) in the context. And that looks like another proof of
bitwise coding inefficiency at this type of data... guess its really the time to start writing
an unary bytewise coder.
Note that the best BWT results on world95.txt from maximumcompression.com
seem to be 467793 from UHBC and 457426 from M03.
Also paq8o6 -7 compresses calgary_bwt to 772989 and calgary_rbwt to 771273,
so there're clearly some possibilities for improvement as paq8's preprocessing tricks
don't apply to BWT output, and its highly improbable that order3+ contexts can be of any help.
o1rc8 | mix(o0',mix(o0,mix(o1',o1))) | 793254
| o1rc8a | o1rc8 with parameters tuned to world95.bwt | 792668
| o1rc9 | mix(mix(o1,o1'),mix(o0,o0')), tuned to calgary.tar.bwt | 787211
| o1rc9a | improved version | 785782
| o1rc9b | cleaned o1rc9a, tuned to calgary.tar.rbwt | 785643
| o1rc9f | further cosmetic changes | 785456
| o1rc9g | mix(mix(mix(o1,o1'),o1''),mix(o0,o0')) | 784037
| o2rcA | mix(o2,mix(mix(o1,o1'),mix(o0,o0')))<.b> | 784956
| o2rcA1 | cleaned o2rcA, tuned to calgary.tar.rbwt | 784684
|
calgary | calgary.tar
| calgary.seg | segmented with Shkarin's seg_file (default level)
| calgary.bwt | using BWT.cpp
| calgary_rbwt | byte reverse + BWT
| calgary_dc1 | Binder's DC 0.99.307: dc -a -d calgary calgary_dc1
| calgary_dc2 | dc -d calgary calgary_dc2 (BWT + all filters enabled, but coding disabled)
| calgary_dc2 | dc -a -d +feodxlb calgary calgary_dc3 (BWT + all filters enabled, but coding disabled)
| calgary_seg_bwt | seg_file + BWT.cpp + separate compression + rar -m0 archive
| calgary_seg_rbwt | seg_file + reverse + BWT.cpp + separate compression + rar -m0 archive
| calgary_seg_dc1 | seg_file + DC's BWT + separate compression + rar -m0 archive
| calgary_seg_dc2 | seg_file + DC's BWT&filters + separate compression + rar -m0 archive
| world95 | world95.txt
| book1 | from calgary.tar
| wcc386 | wcc386.exe
|
Coder:
| source | bcm002 | fpaq0pv4B | o1rc8 | o1rc8a | o1rc9 | o1rc9a | o1rc9b | o1rc9f | o1rc9g | o2rcA | o2rcA1
|
---|
|
---|
BOOK1
| 768771 | 212672 | 441300 | 350899 | 351415 | 348405 | 350784 | 350205 | 350079 | 350014 | 349041 | 348794 |
---|
BOOK1_bwt
| 768771 | | 243630 | 212418 | 212308 | 211708 | 211614 | 211586 | 211393 | 211172 | 211267 | 211256 |
---|
BOOK1_dc1
| 768860 | | 242948 | 212364 | 212310 | 211398 | 211419 | 211348 | 211137 | 210992 | 211021 | 210963 |
---|
BOOK1_dc2
| 544443 | | 238980 | 205887 | 205687 | 205383 | 206673 | 206191 | 206268 | 206011 | 205687 | 205511 |
---|
BOOK1_dc3
| 772091 | | 237730 | 206519 | 206467 | 205660 | 205476 | 205424 | 205253 | 205101 | 205148 | 205117 |
---|
BOOK1_rbwt
| 768771 | | 242878 | 212300 | 212215 | 211299 | 211341 | 211267 | 211047 | 210908 | 210937 | 210871 |
---|
BOOK1_seg_bwt
| 768984 | | 243838 | 212631 | 212545 | 211931 | 211808 | 211787 | 211599 | 211374 | 211472 | 211461 |
---|
BOOK1_seg_dc1
| 769153 | | 243198 | 212583 | 212552 | 211612 | 211611 | 211541 | 211331 | 211188 | 211219 | 211158 |
---|
BOOK1_seg_dc2
| 544736 | | 239216 | 206149 | 205951 | 205608 | 206883 | 206404 | 206484 | 206226 | 205907 | 205729 |
---|
BOOK1_seg_dc3
| 772405 | | 237972 | 206830 | 206720 | 205894 | 205688 | 205638 | 205468 | 205323 | 205360 | 205332 |
---|
BOOK1_seg_rbwt
| 768989 | | 243096 | 212507 | 212392 | 211521 | 211536 | 211464 | 211252 | 211115 | 211133 | 211075 |
---|
|
---|
calgary
| 3152896 | 791434 | 1680338 | 1259218 | 1262575 | 1249383 | 1255225 | 1254212 | 1254275 | 1254908 | 1251319 | 1250665 |
---|
calgary_bwt
| 3152896 | | 981490 | 795650 | 794616 | 789541 | 787668 | 787842 | 787756 | 786387 | 786639 | 786809 |
---|
calgary_dc1
| 3152994 | | 981060 | 793969 | 793145 | 787908 | 786499 | 786381 | 786177 | 784772 | 785662 | 785394 |
---|
calgary_dc2
| 2778210 | | 996936 | 799444 | 798445 | 793799 | 794435 | 793650 | 793798 | 791925 | 792893 | 792723 |
---|
calgary_dc3
| 3167109 | | 975626 | 789337 | 788501 | 783530 | 782035 | 781934 | 781720 | 780359 | 781275 | 781033 |
---|
calgary_rbwt
| 3152896 | | 979796 | 793254 | 792668 | 787211 | 785782 | 785643 | 785456 | 784037 | 784956 | 784684 |
---|
calgary_seg_bwt
| 3154474 | | 971622 | 769050 | 768648 | 764586 | 763643 | 763543 | 763290 | 761969 | 762460 | 762437 |
---|
calgary_seg_dc1
| 3155863 | | 971390 | 767789 | 767292 | 762935 | 762464 | 762199 | 761838 | 760626 | 761131 | 761041 |
---|
calgary_seg_dc2
| 2156735 | | 948284 | 743547 | 742888 | 739230 | 740469 | 739322 | 739346 | 737745 | 738293 | 737770 |
---|
calgary_seg_dc3
| 3157830 | | 965844 | 762759 | 762001 | 757623 | 756979 | 756744 | 756477 | 755277 | 755808 | 755548 |
---|
calgary_seg_rbwt
| 3154511 | | 970564 | 766977 | 766433 | 762086 | 761693 | 761426 | 761067 | 759872 | 760401 | 760247 |
---|
|
---|
wcc386
| 536624 | 282718 | 400420 | 303075 | 305538 | 305179 | 305572 | 305706 | 305827 | 306622 | 305067 | 306063 |
---|
wcc386_bwt
| 536624 | | 315546 | 282822 | 282616 | 281931 | 281598 | 281570 | 281584 | 281117 | 281388 | 281427 |
---|
wcc386_dc1
| 536702 | | 315300 | 282203 | 282055 | 281333 | 281063 | 281009 | 280902 | 280470 | 280821 | 280725 |
---|
wcc386_dc2
| 551470 | | 315098 | 272417 | 272113 | 270794 | 270676 | 270603 | 270549 | 270014 | 270468 | 270348 |
---|
wcc386_dc3
| 536707 | | 315304 | 282208 | 282002 | 281338 | 281068 | 281015 | 280907 | 280474 | 280825 | 280730 |
---|
wcc386_rbwt
| 536624 | | 315256 | 282120 | 281949 | 281278 | 281013 | 280959 | 280852 | 280419 | 280771 | 280675 |
---|
wcc386_seg_bwt
| 537152 | | 308582 | 277535 | 277529 | 276998 | 276671 | 276600 | 276562 | 276138 | 276433 | 276499 |
---|
wcc386_seg_dc1
| 537604 | | 308470 | 277154 | 277179 | 276702 | 276384 | 276313 | 276203 | 275817 | 276122 | 276043 |
---|
wcc386_seg_dc2
| 524852 | | 308180 | 268315 | 268098 | 267051 | 266862 | 266782 | 266695 | 266169 | 266644 | 266586 |
---|
wcc386_seg_dc3
| 536938 | | 308684 | 277369 | 277384 | 276914 | 276597 | 276526 | 276411 | 276020 | 276319 | 276243 |
---|
wcc386_seg_rbwt
| 537165 | | 308200 | 276950 | 276919 | 276449 | 276146 | 276078 | 275963 | 275575 | 275882 | 275802 |
---|
|
---|
world95
| 2988578 | 474298 | 1862146 | 1333942 | 1340040 | 1321571 | 1318986 | 1319200 | 1319125 | 1317509 | 1310936 | 1310059 |
---|
world95_bwt
| 2988578 | | 648752 | 478050 | 475989 | 471773 | 472751 | 472760 | 472574 | 471879 | 471968 | 472046 |
---|
world95_dc1
| 2988672 | | 647194 | 476701 | 474654 | 470068 | 470739 | 470707 | 470571 | 469832 | 469969 | 470201 |
---|
world95_dc2
| 2339029 | | 639862 | 462754 | 461250 | 456848 | 458155 | 457734 | 457757 | 456537 | 457287 | 457026 |
---|
world95_dc3
| 2852581 | | 637804 | 466585 | 464540 | 460243 | 460761 | 460781 | 460619 | 459917 | 460153 | 460282 |
---|
world95_rbwt
| 2988578 | | 647124 | 476693 | 474635 | 469909 | 470578 | 470522 | 470390 | 469640 | 469820 | 470046 |
---|
world95_seg_bwt
| 2989034 | | 667084 | 482645 | 480699 | 476495 | 477246 | 477266 | 477089 | 476399 | 476383 | 476516 |
---|
world95_seg_dc1
| 2989415 | | 665804 | 481004 | 478954 | 474361 | 474856 | 474836 | 474716 | 473918 | 474105 | 474336 |
---|
world95_seg_dc2
| 2360995 | | 661318 | 469762 | 468428 | 463915 | 465021 | 464589 | 464605 | 463347 | 464075 | 463753 |
---|
world95_seg_dc3
| 2854179 | | 656426 | 470939 | 468962 | 464738 | 465086 | 465103 | 464926 | 464205 | 464383 | 464558 |
---|
world95_seg_rbwt
| 2989045 | | 665540 | 480833 | 478723 | 474149 | 474658 | 474644 | 474519 | 473717 | 473912 | 474134
|
---|
2013-08-15 03:50:20 >
2014-11-26 21:12:23 >
2015-01-11 20:10:53 >
|
|