LZRW3 Data Compression Core

1. LZRW3 Data Compression Core Project part A final presentation ShaharZuta NetanelYamin Advisor: Moshe porian Dual semester project April 2013

2. Contents • Project Goals • Top Architecture • Micro Architecture • Problems & Solutions • Work method • Test Plan • Synthesis Results • Part B • Schedule

3. Project Goals • Implementation of LZRW3 data compression algorithm • High performance- data transfer of 1Gbps • Adapted to data templates of 2Kbyte to 32Kbyte • Internal memory on FPGA only ( Virtex-5 ), no interface to external memory • Implementing strong debugging capabilities via GUI

4. LZRW3 algorithm overview OUTPUT FILE [----][-----]-[-------][-----------][----] INPUT FILE ------------------------------------------------------- LZRW3 COMPRESSOR LITERALS ONLY GROUPS OF ITEMS (literal/Copy) A copy item consists of two bytes that represent from 3 to 18 bytes. literal item consist of one byte which represents himself

5. Top Architecture XILINX VIRTEX 5 ON XUVP505 BOARD Rx PATH UART GUI LZRW3 COMPRESSOR CORE INPUT BLOCK memory Tx PATH COMPRESSED FILE memory UART

6. Top Architecture XILINX VIRTEX 5 ON XUVP505 BOARD Rx PATH UART GUI LZRW3 COMPRESSOR CORE INPUT BLOCK memory Tx PATH COMPRESSED FILE memory UART

7. Compressor core – Micro Architecture

8. Problems & solutions

9. Problem # 1 • Hash table (stage III) should not update if we build a copy item. • The decision if a copy item will be built must consider results of the comparison(stage IV) • In the original design we have four clock cycles untill stage IV will have the comparison result but, we have to deal with some cases like ABCABC ABC(index,length) lzrw3

10. 1 Original design clk 2 STAGE IV clk 4 3 Tentative Next Address clk 2 clk STAGE III clk RAM BANKS DATA Address Alignment MEM REG. Hash Table O COMPARATOR UPDATE ENABLE 1

11. Heart of the problem Dealing with cases like ABCABC mean that we need to update the hash table entry with ABC offset very fast. ABC BCA CAB ABC :Stage III samples 1st ABC at Stage III Two clocks only 2nd ABC need to read 1st ABC offset! 2nd ABC at Stage III

12. Solution clk STAGE IV clk Tentative Next Address clk clk STAGE III clk RAM BANKS Address Alignment MEM REG. Hash Table COMPARATOR UPDATE ENABLE 1 ABC offset

13. Problem # 2 • The GOLDEN MODEL does not start building a new copy item if it’s in the last 18 bytes of the file, if it’s in the middle of building a copy item it will finish it.

14. Solution # 1 • Maintain a FIFO buffer that will hold the two last groups, then when end of file is reached we will send the buffered bytes and not compressed bytes. • This solution would demand hardware addition and would demand thorough synchronization which could make the system unstable.

15. Solution # 2 • Taking advantage of the pipeline design: Until the data reaches stage IV (the comparator) it goes through 9 buffers. • Buffer another 9 will complete to 18 clocks buffer before a received data will be compared. • FORWARD the EOF core input signal to stage IV and make it cause automatic comparison failure in the last 18 bytes which will result that no copy items will be created.

16. Implementation 4 5 INPUT FILE memory Hash table Hash Func OUTPUT GROUP INPUT BYTE 9 clocks buffer in the original design When EOFFARWARDUP, NEW COPY ITEMS will not be created GROUP REG INPUT REG HASH REG TABLE REG 3 Bytes buffer End Of File COMPARATOR EOFFARWARD 9 NEW BUFFERS 9 8 7 6 5 4 3 2 1

17. Work method • Generating DIRECT/RANDOM input file. • Compress the input file using the GOLDEN MODEL. • Compress the same input file using our CORE. • Perform a comparison between the two files using DIFF software.

18. Work method (cont.) DIRECT input RANDOM input C CODE ASCII value chars CORE simulation GM C ASCII value ExamDiff

19. ExamDiff

20. Test Plan

21. Test Plan • Basic set: include large amount of tests • Second set: Basic tests + Random client-ready toggles • Third set: Second set + Random inputs valid toggles

22. Basic set examples • Random input (Length, Num Of Vars)

23. Basic set examples • Direct input PERFORM A COPY ITEM COPY ITEM SHOULD NOT PERFORM

24. Basic set examples • Files concatenation BUSY READY FINISH DATA PROVIDER DATA CLIENT LZRW3 CORE C L A E R

25. Synthesis Results

26. Synthesis Results • Timing report Requested frequency : 125 MHz

27. Synthesis Results • Resource utilization :

28. Part B

29. Part B • Assemble lzrw3 core with peripheral units • Logic Test End To End • Burning to FPGA • Hardware test & optimization • Verification Environment - GUI (Visual Studio) • All system Test & Debug

30. Schedule

31. Schedule