The horrors

1. The horrors Nezer (T.A) and Evyatar (Bodek)

2. This ex – what we learned planned For some of you vi or other unix editors make/makefile Debugging code in UNIX (gdb/ddd and core dumps) Opendir/readdir/dirent grep(1) and other basic UNIX functions Strace(1) Working on schedule Ps(1)/kill(1) etc. • Stream sockets – TCP • Stream sockets – UDS • Select, blocking and non blocking system calls • fork(2) • syslog(3) • getenv(3) • time(2) • signal(3)/sigaction(2) • waitpid(2)

3. Some of you were lucky…. • The lucky encountered enough bugs to learn something…. • (The extremely lucky ones lost sleep over some of the bugs so they will remember them!) • This is a collection of common errors we encountered for the less fortunate ones • We also bring some style errors so you can learn for future ex.

4. Busy waiting The code Better ways to act TimeToSleep=atoi(argv[2])-time(NULL); sleep(TimeToSleep); // or alarm(TimeToSleep); pause(); // or struct timeval tv; tv.tv_sec=TimeToSleep; tv.tv_usec=0; select (1, MULL, NULL, NULL, &tv); While (timeToRun>time(null) ; // that is call time infinite number of times

5. The chicken and the egg What comes first > or =? If (pid = fork() < 0)…. How it should be done If ((pid = fork()) < 0) < is calculated before =!!!

6. Logical assumptions and checking error codes A team comes to me with the following input… • They didn’t change the server • The server used to work • Now the server does not appear in ps(1) • It also doesn’t accept clients • Reaching the only logical conclusion • The server becomes invisible • And UNIX hates them After de-mysitification…. • UNIX doesn’t hate them • The server crashes After further investigation we found the crash in following line Myport = atoi(getenv(“My port”)); What lesson have we learned?

7. The OS is my bitch 1 Fopen mania How it should be done At main start : fopen/fopen Each time we log we write/fprint (and possible fflush) At finish we fclose/close x=fopen (“log.txt”,”a”); fprintf(x,”something”); fclose(x); // some lines later x=fopen (“log.txt”,”a”); fprintf(x,”something”); fclose(x);

8. The OS is my bitch 2 Stupid sleeping What is wrong sleep the time difference Don’t use magic numbers No points were deducted for this (this is not busy waiting, just stupid) While (timeToRun>time(null) sleep(1)

9. Garbage is my friend Some of you just had to : Send (sockfd, buf, 100, 0) Resulting in sending 100-strlen(buf) bytes of garbage. Unlucky perhaps but the garbage also contained null so your printing didn’t detect it

10. MY PRECIOUS BUFFER!!!! MINE! MINE!!! • Using buffers returned by dirent structure is perfectly legal…. Yet some of you had to copy everything to YOUR OWN buffer….. • Some of you even malloced the buffer and freed it at every iteration • Some of you forgot to free • Unless there was also no free no points were deducted.

11. The peak Not even 8.3 How it should work Use original dirent data. (no need to copy) Use dirent struct to tell you how much to send Memcpy(buf,dp->name,10); Send(sockfd,s,10,0);

12. Some things just hurt our eyes • Functions should be described in one sentence with no and/or. • If you can’t describe your functions with one sentence – break them to two.

13. How to learn something new!THE KISS METHOD • KISS = KEEP IT SIMPLE STUPID! • Don’t automatically start with a huge program because if it breaks you can’t figure how and why! • Understand from book/man page what you want to do. • Write a small test case for each new system call you learn. – make the test case work. • Make sure you understand how it works then integrate in your main code! • When all else fails…. RTFM = read the fucking friendly manpage!!!!

14. Common errors • Daemon process under SVR4 UNIX (Linux, Solaris, HP-UX) need to fork() twice. (13.3 Advanced programming in the UNIX environment.) • For those that seen other sources in the internet: Daemon process under BSD UNIX (xxxBSD, OSX, AIX) can fork() only once. (but forking twice is not a mistake) • Environment parameters ARE NOT command line argument. (read the UNIX process presentation) • UDS socket was relative path name. (./smt) this will fail if the client is run from different directory! Use absolute path (/tmp/smt or ~/smt) to avoid this.

15. Using Syslog in TAU /* works between processes (such as between bzip and main thread in the ex. Not between threads) */ Int main() { openlog(“scipio”, 0, LOG_LOCAL5); syslog(LOG_ERR, “%s, %d : hello world”, __FILE__, __LINE__); closelog(); }

16. Fcntl file locking example #include <stdio.h> #include <stdlib.h> #include <errno.h> #include <fcntl.h> #include <unistd.h> int main(int argc, char *argv[]) { /* l_type l_whence l_start l_len l_pid */ struct flock fl = { F_WRLCK, SEEK_SET, 0, 0, 0 };

17. Fcntl example cont’d int fd; fl.l_pid = getpid(); if (argc > 1) fl.l_type = F_RDLCK; fd = open("lockdemo.c", O_RDWR) fcntl(fd, F_SETLKW, &fl); fl.l_type = F_UNLCK; fcntl(fd, F_SETLK, &fl) ; close(fd); }

18. GCC versions Char s[200]; Sizeof(s)??? In tau that returns 200; We have tried on other servers and since sizeof char *=4 and char [] is actually char*…. That returned 4. To avoid problems in the future don’t do sizeof array but sizeof (element) * array size. No points were deducted. (Evyatar apologizes for causing several undue heart attacks).

19. Request from System • Avoid running your jobs on NOVA if you can • You are • Running lots of jobs • Busy waiting • Taking all the CPU • Instead once you login, ssh to any doppler or Euler workstation and work there. • NOVA is the only host accessible from outside TAU. So if you login to NOVA and load it with unbearable load you are screwing your fellow students

20. IF NOVA DOESN’T REPLY • Don’t wait for the last minute to submit the ex. • Check your timeouts. NOVA was very stable during the ex. Submission but also very loaded and several students failed to login due to short timeouts

21. REFERENCES FOR USERLAND • MAN PAGES • ADVANCED PROGRAMMING IN THE UNIX ENVIRONMENT/Stevens & Rago (chapters 4,7,8,9,10,13,14,(15 = bonus), 16, 17 were covered in ex. 1 + chapters 3+8+10+11+12+14 are covered in ex. 2) • KR (The C programming language – Brian Keringhan and Dennis Ritchie) Other good books • Solaris System programming / Rich teer (run the examples to make sure they work on Linux as well) 1,4,6,10,13,14,15,16,17,18,19,21 – Almost identical arrangement to Stevens with some Solaris specific code • Advanced UNIX programming (Warren W. Gay) 1-7, 9-11, 15-19,22,26

22. The Cost • send(3) with fixed buflen parameter: -3 • Segmentation fault when running client with missing/no parameters: -5 • UDS only connects when client and server are in the same folder: -10 • Sending only 10 chars of file name. -10 • Not using environment variables at all: -10 • Using printf() in the server daemon: -2 • Having us kick-start your project (i.e. project that would crash) – variable

23. More cost • Not printing daemon getppid(): -10 (first line in the server description!) • Not setting the session ID and letting me use ctrl+c to kill your server: -10 • Client hangs on a past/illegal time index: -5 • Server crashes when UDS connects: -30 • Not logging to syslog() OR file: -10 • Not running as Daemon at all. -20 • Not logging child termination: -10

24. A Silver Lining • Leaving defunct processes (zombies): just a warning • Printing getpid() instead of getppid(): just a notice • Non-functional FIFO with effort: +[1,9] • Doing well on the oral exam: +[1,10]

25. FYI • With the exception of one team (who had some problems) all teams submitted on time. • AVG grade was about 85. • No students that was tested Wednesday copied his work in our opinion • Most students were very successful in the oral exam. (nobody was mistaken in more then 1 question. And 1 was rare)

26. Regarding ex. 2 • Here is what I meant for you to do 1. I am a thread 2. The file to be sent is X, last position sent was Y, chunk size is Z 3. If the file isn't locked:       3.1. lock       3.2 create chunk from Y to Y+Z   3.3 Increase Y by Z       3.4 Release lock 3.5 send (Send can also be done in 3.2 but this is more accurate) Thanks to Dor Levi that put it clear.

27. Defining protocol • If you want to support only 1 client you only need to define the last packet and the packet number (As they may arrive out of order). You may want to transfer the filesize and offset instead. • If you want to support several clients you need to add mutex for file name received (or to start different set of connection handling threads for each client, with different conds) and make sure files don’t overwrite each other. • Exact design it up to you. You may use temp files but you have to clean your work

28. Syncing • While it is possible to do no syncing AT ALL (this ex. Can work even as single thread sending or compressing. (since we work on the same host no sending is even necessary) • BUT you are required to sync the file reading and sending process