# 236601 - Coding and Algorithms for Memories Lecture 5 - PowerPoint PPT Presentation Download Presentation 236601 - Coding and Algorithms for Memories Lecture 5

236601 - Coding and Algorithms for Memories Lecture 5 Download Presentation ## 236601 - Coding and Algorithms for Memories Lecture 5

- - - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - - -
##### Presentation Transcript

1. 236601 - Coding and Algorithms for MemoriesLecture 5

2. Write-Once Memories (WOM) • Introduced by Rivest and Shamir, “How to reuse a write-once memory”, 1982 • The memory elements represent bits (2 levels) and are irreversibly programmed from ‘0’ to ‘1’ Q:How many cells are required to write 100 bits twice? P1: Is it possible to do better…? P2: How many cells to write kbits twice? P3: How many cells to write kbits ttimes? P3’: What is the total number of bits that is possible to write in n cells in t writes? 1st Write 2nd Write

3. Definition: WOM Codes • Definition: An [n,t;M1,…,Mt] t-write WOM codeis a coding scheme which consists of n cells and guarantees any t writes of alphabet size M1,…,Mt by programming cells from zero to one • A WOM code consists of t encoding and decoding maps Ei, Di, 1 ≤i≤ t • E1: {1,…,M1}  {0,1}n • For 2 ≤i≤ t, Ei: {1,…,Mi}×{0,1}n {0,1}nsuch that for all (m,c)∊{1,…,Mi}×{0,1}n, Ei(m,c) ≥ c • For 1 ≤i≤ t, Di: {0,1}n {1,…,Mi}such that for Di(Ei(m,c)) =m for all (m,c)∊{1,…,Mi}×{0,1}n • The sum-rate of the WOM code is R = (Σ1t logMi)/n RivestShamir: [3,2;4,4], R = (log4+log4)/3=4/3

4. The Capacity of WOM Codes • The Capacity Region for two writes C2-WOM={(R1,R2)|∃p∊[0,0.5],R1≤h(p), R2≤1-p} h(p) – the entropy functionh(p) = -plog(p)-(1-p)log(1-p) • p – the prob to program a cell on the 1st write, thus R1≤ h(p) • After the first write, 1-p out of the cells aren’t programmed, thus R2≤1-p • The maximum achievable sum-rate ismaxp∊[0,0.5]{h(p)+(1-p)} = log3 achieved for p=1/3: R1 = h(1/3) = log(3)-2/3 R2= 1-1/3 = 2/3

5. The Capacity of WOM Codes • The Capacity Region for two writes C2-WOM={(R1,R2)|∃p∊[0,0.5],R1≤h(p), R2≤1-p} h(p) – the entropy functionh(p) = -plog(p)-(1-p)log(1-p) • The Capacity Region for t writes: Ct-WOM={(R1,…,Rt)| ∃p1,p2,…pt-1∊[0,0.5], R1≤ h(p1), R2 ≤ (1–p1)h(p2),…, Rt-1≤ (1–p1)(1–pt–2)h(pt–1) Rt≤ (1–p1)(1–pt–2)(1–pt–1)} • p1- probto prog. a cell on the 1st write: R1≤ h(p) • p2- probto prog. a cell on the 2nd write (from the remainder): R2≤(1-p1)h(p2) • pt-1- prob to prog. a cell on the (t-1)thwrite (from the remainder): Rt-1≤(1–p1)(1–pt–2)h(pt–1) • Rt≤ (1–p1)(1–pt–2)(1–pt–1) because(1–p1)(1–pt–2)(1–pt–1) cells weren’t programmed • The maximum achievable sum-rate is log(t+1)

6. James Saxe’s WOM Code • [n,n/2-1; n/2,n/2-1,n/2-2,…,2] WOM Code • Partition the memory into twoparts of n/2 cells each • First write: • input symbol m∊{1,…,n/2} • program the ith cell of the 1stgroup • The ith write, i≥2: • input symbol m∊{1,…,n/2-i+1} • copy the first group to the second group • program the ith available cell in the 1stgroup • Decoding: • There is always one cell that is programmed in the 1st and not in the 2nd group • Its location, among the non-programmed cells, is the message value • Sum-rate: (log(n/2)+log(n/2-1)+ … +log2)/n=log((n/2)!)/n≈ (n/2log(n/2))/n ≈ (log n)/2

7. The Coset Coding Scheme • Cohen, Godlewski, and Merkx ‘86 – The coset coding scheme • Use Error Correcting Codes (ECC) in order to construct WOM-codes • Let C[n,n-r] be an ECC with parity check matrix H of size r×n • Write r bits: Given a syndrome s of r bits, find a length-n vector e such that H⋅eT = s • Use ECC’s that guarantee on successive writes to find vectors that do not overlap with the previously programmed cells • The goal is to find a vector e of minimum weight such that only 0s flip to 1s

8. The Coset Coding Scheme • C[n,n-r] is an ECC with an r×nparity check matrix H • Write r bits: Given a syndrome s of r bits, find a length-n vector e such that H⋅eT = s • Example: H is the parity check matrix of a Hamming code • s=100, v1 = 0000100: c = 0000100 • s=000, v2 = 1001000: c = 1001100 • s=111, v3 = 0100010: c = 1101110 • s=010, …  can’t write! • This matrix gives a [7,3:8,8,8] WOM code

9. Binary Two-Write WOM-Codes • First Write: program only vectors v such that rank(Hv) = rVC = { v ∊ {0,1}n | rank(Hv) = r} • For H we get |VC| = 92 - we can write 92 messages • Assume we write v1= 0 1 0 1 1 0 0 v1= (0 1 0 1 1 00)

10. Binary Two-Write WOM-Codes • First Write: program only vectors v such that rank(Hv) = r, VC = { v ∊ {0,1}n | rank(Hv) = r} • Second Write Encoding: • Second Write Decoding: Multiply the received word by H: H⋅(v1 + v2) = H⋅v1 + H⋅v2 = s1+ (s1 + s2) = s2 • s2 = 001 • s1 = H⋅v1 = 010 • Hv1⋅v2 = s1+s2 = 011a • v2 = 0 0 0 0 0 1 1 • v1+v2 = 0 1 0 1 1 1 1 • Write a vector s2 of r bits • Calculates1=H⋅v1 • Find v2 such that Hv1⋅v2= s1+s2a • v2 exists since rank(Hv1) = ra • Write v1+v2 to memory v1= (0 1 0 1 1 00)

11. Sum-rate Results • The construction works for any linear code C • For any C[n,n-r] with parity check matrix H, VC = { v ∊ {0,1}n | rank(Hv) = r} • The rate of the first write is: R1(C) = (log2|VC|)/n • The rate of the second write is: R2(C) = r/n • Thus, the sum-rate is: R(C) = (log2|VC| + r)/n • In the last example: • R1= log(92)/7=6.52/7=0.93, R2=3/7=0.42, R=1.35 • Goal: Choose a code C with parity check matrix H that maximizes the sum-rate

12. Capacity Achieving Results • The Capacity region C2-WOM={(R1,R2)|∃p∊[0,0.5],R1≤h(p), R2≤1-p} • Theorem: For any (R1, R2)∊C2-WOM and ε>0, there exists a linear code C satisfying R1(C) ≥ R1-εand R2(C) ≥ R2–ε • By computer search • Best unrestricted sum-rate 1.4928 (upper bound 1.58) • Best fixed sum-rate 1.4546 (upper bound 1.54)

13. Capacity Region and Achievable Rates of Two-Write WOM codes

14. Non-Binary WOM Codes • Definition: An [n,t; M1,…,Mt]q t-write WOM code is a coding scheme that consists of n q-ary cells and guarantees any t writes of alphabet size M1,…,Mt only by increasing the cell levels • The sum-rate of the WOM-code is R = (Σi=1tlogMi)/n

15. WOM Capacity • The capacity of non-binary WOM-codes was given by Fu and Han Vinck, ‘99 • The maximal sum-rate using t writes and q-ary cells is • There is no tight upper bound on the sum-rate in case equal amount of information is written on each write

16. Basic Constructions • Construction 1: Assume t=q–1and there are nq-ary cells • On each write, n binary bits are written • Write to the cells level by level • The sum-rate is n(q–1)/n = q-1 (upper bound ≈ 2(q-1)) • Construction 2: Assume t=2 and q is odd • First write: use only levels 0,1,…,(q-1)/2 • Second write: use only levels (q-1)/2,…,q-1 • The sum-rate is 2log((q+1)/2) • The difference between the upper bound is

17. Construction 3 • Assume q is a power of 2, q=2m • Every cell is converted into m bits • For 1 ≤ i ≤ m, the ith bits from each cell comprise a t-write binary WOMcode • The m WOM codes write into their corresponding bits in the cells independently • Since every bit can only change from 0 to 1, the level of each cell cannot decrease • Use the binary expansion ϕ:{0,1}m→{0,…,2m-1} x=(x0,…,xm-1),ϕ(x) = ∑j 2m-j xj

18. Example – Construction 3 • Let q = 8 = 23 and n = 3 • Every cell corresponds to 3 bits • Use Rivest-Shamir WOM-code • The ith bits from each cell comprise a WOM-code • The sum-rate is 6∙2/3 = 4 (upper bound is 5.17)

19. Construction 3 • Theorem: If q=2m and there exists a t-write binary WOM-code of sum-rate R, then there exists a t-write q-ary WOM-code of sum-rate mR

20. Binary Two-Write WOM-Codes • First Write: program only vectors v such that rank(Hv) = r, VC = { v ∊ {0,1}n | rank(Hv) = r} • Second Write Encoding: • Second Write Decoding: Multiply the received word by H: H⋅(v1 + v2) = H⋅v1 + H⋅v2 = s1+ (s1 + s2) = s2 • s2 = 001 • s1 = H⋅v1 = 010 • Hv1⋅v2 = s1+s2 = 011a • v2 = 0 0 0 0 0 1 1 • v1+v2 = 0 1 0 1 1 1 1 • Write a vector s2 of r bits • Calculates1=H⋅v1 • Find v2 such that Hv1⋅v2= s1+s2a • v2 exists since rank(Hv1) = ra • Write v1+v2 to memory v1= (0 1 0 1 1 00)

21. Non-Binary Two-Write WOM-Codes • Let C[n,n-r] be a linear code w/ p.c.mH of size r×n over GF(q) • For a vector v ∊ GF(q)n, let Hv be the matrix H with 0’s in the columns that correspond to the positions of the nonzeros’s in v • First Write: program only vectors v such that rank(Hv)=r, VC = { v ∊GF(q)n | rank(Hv) = r} • Second Write Encoding: Write a vector s2 of rsymbols overGF(q) • Let v1 be the programmed vector on the first write, and let s1=H⋅v1 • Remember that rank(Hv1) = ra • Find v2 such that Hv1⋅v2= -s1+s2F • A solution v2 exists since rank(Hv1) = ra • Write v1+v2 to memory • Second Write Decoding: Multiply the received word by H: H⋅(v1 + v2) = H⋅v1 + H⋅v2 = s1+ (-s1 + s2) = s2 • This construction works for any linear code over GF(q)

22. Code Rate • The construction works for any linear code C over GF(q) • For any C[n,n-r] with parity check matrix H,VC = { v ∊GF(q)n | rank(Hv) = r} • The rate of the first write is: R1(C) = (log2|VC|)/n • The rate of the second write is: R2(C) = r⋅log2q/n • Thus, the total rate is: R(C) = (log2|VC| + r⋅log2q)/n • By choosing the parity check matrix H uniformly at random, it is possible to achieve the following capacity region: {(R1,R2)|∃p∊[0,1-1/q], R1≤h(p)+p⋅log2(q-1), R2≤(1-p)⋅log2q} • The maximum achievable rate log2(2q-1) is achieved for p=(q-1)/(2q-1) • Remarks: • This is not an optimal non-binary two-write WOM code construction • Cells cannot be programmed twice • However, these codes are useful to construct multiple-write WOM codes

23. Three-Write WOM Codes • Let C3 be an [n,2; 2nR1,2nR2] two-write WOM code over GF(3) • Construct a binary [2n,3; 2nR1,2nR2,2n]3-writeWOM code as follows: • The code has 2n cells, which are broken into n2-cell blocks • First write: use the first write of the code C3 • Write a message m1 from {1,…, 2nR1} • This message is supposed to be written as a ternary vector v1 ∊ GF(3)n • Write the vector v1 into the n 2-cell blocks, using the mapping Φ :GF(3)→ (GF(2),GF(2)),where Φ(0)=(0,0), Φ(1)=(1,0),Φ(2)=(0,1) • Second write: use the second write of the code C3 • Write a message m2 from {1,…, 2nR2} • The ternary vector v2 is written into the n 2-cell blocks using the mapping Φ • Each 2-cell block is written at most once and at most one cell is programmed m1∊{1,…, 2nR1} C3 Encoder v1∊GF(3)n u1∊{0,1}2n Φ m2∊{1,…, 2nR2} v2∊GF(3)n u2∊{0,1}2n C3 Encoder Φ

24. Three-Write WOM Codes • Let C3 be an [n,2; 2nR1,2nR2] two-write WOM code over GF(3) • Construct a binary [2n,3; 2nR1,2nR2,2n]3-writeWOM code as follows: • The code has 2n cells, which are broken into n2-cell blocks • First write: use the first write of the code C3 • Write a message m1 from {1,…, 2nR1} • This message is supposed to be written as a ternary vector v1 ∊ GF(3)n • Write the vector v1 into the n 2-cell blocks, using the mapping Φ :GF(3)→ (GF(2),GF(2)),where Φ(0)=(0,0), Φ(1)=(1,0),Φ(2)=(0,1) • Second write: use the second write of the code C3 • Write a message m2 from {1,…, 2nR2} • The ternary vector v2 is written into the n 2-cell blocks using the mapping Φ • Each 2-cell block is written at most once and at most one cell is programmed • Third Write: write n bits • Each bit is stored in a 2-cell block: the bit value is 1iff both cells are programmed • It is possible to write a bit in each block since at most one cell in each block was programmed

25. Three-Write WOM Code - Example • Ex: C3 is a 2-write WOM code over GF(3) with n=7 cells • First Write • Write a vector v1 over GF(3) according to the first write of C3 • Assume the vector is v1 = 2 0 1 0 2 1 0 • Using the mapping Φ, v1 is written into the 2n = 14 binary cells 2 0 1 0 21 0→Φ 01.00.10.00.01.10.00 • Second Write • Write a vector v2 over GF(3) according to the second write of C3 • Assume the vector is v2 = 2 1 1 0 2 1 2 • Using the mapping Φ, v2 is written into the 2n = 14 binary cells 2 1 1 0 2 1 2 →Φ01.10.10.00.01.10.01 • Third Write: Write 7 bits • Assume the message is v3 = 0 11 0 1 0 1 , then write to memory 01.11.11.00.11.10.11

26. Code Rate • Theorem: Let C3 be an [n,2; 2nR1,2nR2]2-write WOM code over GF(3) of sum-rate R1+R2, then there exists a binary [2n,3; 2nR1,2nR2,2n]3-write WOM code of sum-rate (R1+R2+1)/2 • The best sum-rate for the code C3 is log5 = 2.32 • Conclusion: It is possible to find a binary 3-write WOM code of sum-rate (log5 +1)/2 = 1.66 • Using a computer search, the best sum-rate found for WOM codes over GF(3) is 2.22 and thus there exists a 3-write WOM code of sum-rate 1.61

27. Four-Write WOM Codes • Let C3 be an [n,2; 2nR3,1,2nR3,2] two-write WOM code over GF(3)and letC2be an[n,2; 2nR2,1,2nR2,2]binary two-write WOM code • Construct a [2n,4; 2nR1,2nR2, 2nR2,1,2nR2,2]4-writeWOM code as follows: • The code has 2n cells, which are broken into n2-cell blocks • First write: use the first write of the code C3 • Write a message from {1,…,2nR3,1} • This message is supposed to be written as a ternary vector v ∊ GF(3)n • Write the vector v into the n 2-cell blocks, using the mapping Φ :GF(3)→ (GF(2),GF(2)),where Φ(0)=(0,0), Φ(1)=(1,0),Φ(2)=(0,1) • Second write: use the second write of the code C3 • Write a message from {1,…,2nR3,2} • The ternary vector is written into the n 2-cell blocks using the mapping Φ • Note that each 2-cell block is written only once • Third and fourth writes: use the first and second writes of C2 • Each bit is stored in a 2-cell block: the bit value is 1iff both cells are programmed • It is possible to write a bit in each block since at most one cell in each block was programmed

28. Code Rate • Theorem: Let C3 be an [n,2; 2nR3,1,2nR3,2]2-write WOM code over GF(3) (found before) of sum-rate R1+R2, and letC2be an[n,2; 2nR2,1,2nR2,2] binary two-write WOM codethen there exists a binary [2n,4; 2nR3,1,2nR3,2,2nR2,1,2nR2,2]4-write WOM code of sum-rate (R3,1+R3,2+R2,1+R2,2)/2a • The best sum-rate for the code C3 is log5 = 2.32 and for the codeC2islog3 = 1.58 • Conclusion: It is possible to find a binary 3-write WOM code of sum-rate (log5 + log3)/2 = 1.95 • Using a computer search, the best sum-rate found for codes over GF(3) is 2.22and for codes over GF(2) is 1.49. Thus there exists a 4-write WOM code of sum-rate 1.855

29. Five-Write WOM-Codes • Use a 2-write WOM code C3 over GF(3) of sum-rates R3,1, R3,2 and a binary 3-write WOM code C2 of sum-rates R2,1, R2,2, R2,3a • The first and second writes are implemented as before using the first and second writes of the code C3 • The third, fourth, and fifth writesare also implemented as before by using the first, second, and third writes of C2 • It is possible to achieve a five-write WOM code of rate (log5+(log5+1)/2)/2=1.99 and there exists a code of rate 1.915 • For six writes, we will change the code C2 to be a 4-write WOM code such that it is possible to achieve WOM-rate (log5+1.95)/2 = 2.14 and there exists a code of rate 2.0375