390 likes | 399 Vues
Learn about setting up wireless video conferencing systems, communications protocols, video compression, and picture-in-picture display.
E N D
Wireless Video Conferencing EECS150 Spring 2007 - Lab Lecture #10 Neil Warren Allen Lee EECS150 Lab Lecture #8
Checkpoint #3 Information(1) • Check off Requirement: • #1) Be able to set 2 sets different source and destination addresses on 2 boards and communicate. (CP4 video streaming) • #2) Be able to set both sides’ destination address to 0xFF and a unique source address on each and have communications occur. (CP4 Handshake) EECS150 Lab Lecture #8
Checkpoint #3 Information • If you’re behind, you can forfeit checkpoint #3 and receive a black box. • You will receive a 0 for CP3 but will be able to move on to CP4 which is worth many more points. • Talk to Neil if you want this option. • You lose the points before you get the BB. EECS150 Lab Lecture #8
Welcome to Checkpoint 4 • Combine CP2 and CP3 to make two-way wireless video conferencing system • Note: Specification has changed to make your life much easier – based on questions from students so far. Make sure you work with the non-draft version of the Checkpoint4.doc EECS150 Lab Lecture #8
Overview • Revisit CP2 and CP3 • Communications protocol • Video compression • Picture-in-picture display EECS150 Lab Lecture #8
Recall from Checkpoint 2 EECS150 Lab Lecture #8
Checkpoint 4 Datapath EECS150 Lab Lecture #8
Our Advice (1) • Separate the work between you and your partner to make the best use of your time. • Communications System • Graphics System • Agree on an interface and make sure you conform to it. EECS150 Lab Lecture #8
Our Advice (2) • Debug Independently! • Communications System: • Assume that the graphics system works perfectly (conforms to your interface). • Hardcode in data and requests using timers and debug the system by using the packet sniffer. • Graphics System: • Assume that the communications system works perfectly. • Hardcode transmission inputs, and whatever interface method you use for send/receive acknowledgements. • Much easier to debug each piece, then just put the 2 working pieces together. • If you conform to your interface it should work! EECS150 Lab Lecture #8
Communications Module (1) • Responsibilities • Maintain communications link. • Lost packet robustness • Provide cross-compatibility • Conform to communications protocol • Encode and Decode packets and pass data to graphics engine • Handle the initial handshake between master and slave. EECS150 Lab Lecture #8
Communications Module (2) • Recommended Pieces: • State Machine • Keeps track of the current state of the communications module. • Handles handshaking and properly re-synchronizing the communications link after lost packet. • Sends commands to Packet Encoder, receives updates from Packet Decoder • Packet Encoder • Generate payload data for each type of packet and handle handshake with transceiver. • Take commands from state machine. • Packet Decoder • Passes data to the graphics engine and gives status updates to the state machine. EECS150 Lab Lecture #8
Communications Module (3) • Initial Handshaking • Purpose: • To allow any 2 boxes without knowledge of the other’s address to communicate. • To set up the timing of master and slave send and receives. Note: At the beginning both the master and slave begin with their destination addresses being 0xFF and their source addresses being unique EECS150 Lab Lecture #8
Communications Module (4) • Handshake Method: • Master broadcasts “Master-Seek” packets once every 2.4ms. When a slave receives a “Master-Seek” packet it sends a “Slave-ReqAck” • When the master receives the “Slave-ReqAck” packet it changes its destination address to the source address of the slave and then transmits a “Master-Ready” packet with the new destination address. • When the slave receives a “Master-Ready” packet the slave sends a “Slave-Ready” packet and moves into a state where it can receive active video packets from the master. The “Slave-Ready” packet should be sent with its destination address as the source of the address of the master. • When the master receives the “Slave-Ready” ready packet, it sends its first packet of active video data. • Now the communication should alternate between the master and slave transmitting active video data. EECS150 Lab Lecture #8
Communications Module (5) • Communications Protocol • Purpose: • To allow cross-compatibility between different implementations. • Abstracts away the communications aspect of the project for the graphics module. • The 8-bit header designates the type of packet. • The 8-bit line number indicates which line the respective active video data corresponds to. • The 80-bit data contains the AV data for 32 pixels. (16 pixel pairs) • 8 bit headers and line numbers make it easy to view on the packet sniffer! EECS150 Lab Lecture #8
Communications Module (6) EECS150 Lab Lecture #8
Communications Module (7) • Time Outs – let you know if a communication loss has occurred. • After the expiration of 10ms from the last AV packet transmission, if the slave not does receive a packet it should return to a state where it can receive AV packets from the master. • If this occurs 25 consecutive times without ever receiving an AV packet, then a communication loss is declared and the system must be reset. A message should appear on screen indicating a communication loss has occurred. • After the expiration of 10ms from the last AV packet received, if the master does not receive a packet it should send its next AV packet. • If this occurs 25 consecutive times without ever receiving an AV packet, then a communication loss is declared and the system must be reset. A message should appear on screen indicating a communication loss has occurred. EECS150 Lab Lecture #8
Communications Module (8) • Tips: • Make use of the packet sniffer to determine what is *actually* flying through the air. • The communications protocol was re-designed specifically for this purpose. EECS150 Lab Lecture #8
Video Compression (1) • Wireless transfer is the bottleneck • Uncompressed video: • 507x720x16 = 5,840,640 bits per frame • That’s ~175 Mbits per second! • Theoretical max bandwidth for wireless is 256kbps, so ~22.8 seconds per frame. EECS150 Lab Lecture #8
Video Compression (2) • Simple lossy compression algorithm • Resolution: 176x127 • 5 bits grayscale per pixel pair EECS150 Lab Lecture #8
Video Compression (3) • Compressed Video: • 176x127x5/2 = 55,880 bits per frame • Approximately 105:1 compression • At theoretical max wireless bandwidth, that’s 4.5 frames per second EECS150 Lab Lecture #8
Video Compression (4) • Vertical compression • Sample every other odd active line • (254 active odd lines)/2 = 127 lines • Horizontal compression • Sample every fourth pixel pair (except for the last eight) • (360-8)/4 = 88 pixel pairs = 176 pixels • For each sampled pixel pair, keep only the upper five bits of the average luminance EECS150 Lab Lecture #8
Video Compression (5) • Why 176x127 instead of 180x127? • Good question. There’s a good answer, figure it out! EECS150 Lab Lecture #8
Video Decompression • Reverse of compression, but lossy due to lack of chrominance • 5-bit grayscale represents a pixel pair • Decompresses to 32 bits, with appropriate padding for chrominance values EECS150 Lab Lecture #8
Writing Compressed Frames • Set aside an area in SDRAM for sending and receiving compressed frames • Update the stored frame to send only after the entire frame has been sent • It is okay to lose a local frame to store a compressed one EECS150 Lab Lecture #8
Writing Compressed Frames • Sequence for handling compressed frames: • Wait for SDRAM and ChipCon initialization • Store a compressed frame • Tell transmitter FSM that frame in SDRAM is ready • Wait as compressed frame is sent • Receive signal from transmitter that last packet for frame has been sent • Once entire compressed frame is sent, store a new compressed frame EECS150 Lab Lecture #8
Writing Compressed Frames • Two options for implementing compressed video controller: • Merge with VD processor, using the same FIFO • Advantage: Arbiter sees one write port for local video • Drawback: Need to cross clock boundaries • Separate module with own FIFO • Advantage: May not need to cross clock boundaries • Drawback: Arbiter sees two write ports for local video • Pick the one that makes more sense to you EECS150 Lab Lecture #8
Dual-Clock Communication • Modules running on the camera clock and the system clock cannot safely communicate with direct control signals. • If your modules need to communicate but are on different clocks, you must either implement a safe handshaking protocol or use the memory element we provide. • For your convenience, we’ve included a dual-clocked one-bit-wide FIFO to safely exchange control signals, but you don’t have to use it. EECS150 Lab Lecture #8
Sender and Receiver FSMs • Sender/Transmitter • Waits for frame to be available in SDRAM • Reads from SDRAM and packages data into a packet • Requests a new frame once last packet for frame is sent • Receiver • Decodes the compressed data • Writes received data into SDRAM EECS150 Lab Lecture #8
Displaying Compressed Video EECS150 Lab Lecture #8
Displaying Compressed Video • Constantly display compressed frames to avoid SDRAM decay • This way, you don’t have to auto-refresh! EECS150 Lab Lecture #8
Multi-Port Arbiter • At least two more ports are needed for the sender and receiver FSMs. • A fifth port may be needed depending on how you handle writing compressed frames. EECS150 Lab Lecture #8
Tips, Hints, & Common Pitfalls • Start early! • If you’re already done with CP3, start now! • We are not providing any Verilog besides the optional one-bit-wide memory element. • Run the TA solution to see what the final result should look like. EECS150 Lab Lecture #8
Tips, Hints, & Common Pitfalls • Split up the project into graphics and communications. • These parts are fairly independent so you and your partner can work in parallel. • Do not combine until you’ve verified each part works independently. • It can take up to 25 minutes to push to board after combining! EECS150 Lab Lecture #8
Tips, Hints, & Common Pitfalls • Work Incrementally • Display a sectioned screen • Display only compressed video at full frame rate • Simultaneously display local compressed and uncompressed video at full frame rate • Display uncompressed video at full frame rate and local compressed at 1 FPS • Test sender and receiver FSMs by sending local “packets” to yourself EECS150 Lab Lecture #8
Tips, Hints, & Common Pitfalls • Be careful how you handle the half-full packet (packet 6): • Make sure the receiver knows which half of the payload to look at for the video data EECS150 Lab Lecture #8
Tips, Hints, & Common Pitfalls • 2 Clocks with phase offset: • Camera Clock != Normal Clock • Use the 1-bit wide dual-clocked fifo we provide to cross clock boundaries. EECS150 Lab Lecture #8
Requirements • Must be TA Solution Compatible. • EC can be non-compatible, but you must still develop a compatible solution first. • 1 bit file. Both master and slave systems should contained on 1 bit file. • Onscreen prompts. Should display to the user the communications status (see TA solution for more info) • Some method of outputting the (max) time between new frames. EECS150 Lab Lecture #8
Extra Credit • Maximum 20% • Early check off 10% • Some Ideas (EC% TBD): • Faster frame rate • Larger packet sizes • Faster transmission • Non-full duplex communications • UI options • Menus • Video output modifications (change contrast with buttons, etc.) • Use N64 controller for extra buttons to add functionality • We will provide a black box to interface • Open-ended! Come up with something special, cool, clever, unique, and/or challenging. EECS150 Lab Lecture #8
Questions? EECS150 Lab Lecture #8