1 / 24

UK CMG 2008

UK CMG 2008. H. W. "Barry" Merrill, PhD Merrill Consultants Dallas, TEXAS www.mxg.com Tuesday May 20, 2008 Remote from Emerald Island, NC. A. MXG Support for SAS Version 9.2 Compatible, No ERRORs, New Warnings B. MXG and WPS 1. Current Status of MXG and WPS 2. Run Time Comparison

pembroke
Télécharger la présentation

UK CMG 2008

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. UK CMG 2008 H. W. "Barry" Merrill, PhD Merrill Consultants Dallas, TEXAS www.mxg.com Tuesday May 20, 2008 Remote from Emerald Island, NC

  2. A. MXG Support for SAS Version 9.2 • Compatible, No ERRORs, New Warnings • B. MXG and WPS • 1. Current Status of MXG and WPS • 2. Run Time Comparison • 3. Revisions to SAS Clone Newsletter Article • 4. Summary • 5. QA Steps successfully compiled and run • C. CPU Parked Time Metric • D. zIIP and zAAP Metrics • E. Daily CPU cost of Performance Measurement

  3. A. Support for SAS Version 9.2: COMPATIBLE, no ERRORS, new WARNings All MXG Versions execute WITHOUT error with SAS Version V9.2. V9.2 libraries can be read/written by SAS V8.2 or V9.1.3, & vice versa. SAS V9.2 Phase I Foundation Level z/OS + ASCII was tested. These old MXG Versions WILL print a new SAS V9.2 WARNING, that sets CC=4 (condition/return code), but that warning is harmless (to MXG code), and all MXG output SAS datasets are correct, even with that warning and return code. MXG Version 26.02 eliminates this SAS V9.2 WARNING internally, by enabling new OPTION VARLENCHK=NOWARN to suppress the creation of both the warning and the condition code. So the ONLY exposure with older MXG Versions under V9.2 Is only on z/OS, and ONLY if condition code tests are used in your MXG job streams.

  4. This new-in-SAS V9.2 "MULTIPLE LENGTHS OF A VARIABLE“ • Warning message surfaced in MXG delivered code in two cases: • The shortening of the LENGTH of a numeric variable, but only if • when the LENGTH statement precedes SET/MERGE/UPDATE. • This occurs in VMXGSUM where the fixed-length-8 variables • output by PROC MEANS were reduced to 4-bytes, prior to option • KEEPLEN. The VMXGSUM utility is used in all MXG summarization, • like ASUMxxxx and TRNDxxxx, many ANALxxxx members, and • in summarizing RMFINTRV and CICINTRV programs in BUILDPDB. • It is pervasive in MXG. This case was eliminated in MXG 26.02 by • relocating its LENGTH statement to after the SET. • - The JOIN of multiple datasets (SET MON.JOBS TUE.JOBS ...) • where a variable has different lengths in different datasets, and • the first dataset’s variable has shorter length. • This also occurs in VMXGSUM, when multiple input datasets are • combined, like TRENDing, where TREND had short LENGTHs, • but the "NEWTREND" internally has fixed, longer LENGTHS. • MXG 26.02 adds KEEPLEN option to PROC MEANS to eliminate.

  5. MXG Version 26.02 code eliminates the SAS V9.2 WARNING internally, but also enables OPTION VARLENCHK=NOWARN to suppress BOTH the warning and the condition code, for all your SAS programs, as long as you use the MXGSASV9/CONFIGV9 MXG initialization. Without VARLENCHK=NOWARN, EVEN with MXG 26.02, WARNING can OCCUR: - If you have tailoring members in "USERID.SOURCLIB" from old MXG versions, that need the same code revisions. - In any user-written SAS programs, but this could actually be a valid warning that a variable was unintentionally truncated. or, at any time in the future, the WARNING can still occur: - When an MXG Version that changed a variable LENGTH is installed, subsequent WEEKLY or MONTHLY jobs create the WARNING because some PDB's have the old length and some have the new length, when those multiple datasets are joined. Previous to V9.2, length could be changed with no WARNING. Between MXG 24.24 and 25.25 1206 variables were changed. Hot Fix VARLENCHK for Phase I will be included in Phase II. Changes 26.060 and 26.065 have all the V9.2 details.

  6. B.MXG and WPS 1. Current status of MXG testing under WPS. • What has been tested successfully: • MXG QA compile-only (dummy INFILEs) TYPExxxx and TYPSxxxx members • This exercised the MXG DATA-step Code for compile errors, • and created DOCVER to compare the contents of WPS-built • datasets (variables, LENGTHs, LABELs, FORMATs). • Steps 1 thru 36 of the QAJOBXX were tested. • Default BUILDPDB with 480MB SMF File. • Tailored BUILDPDB with IMACEXCL with 480 MB SMF File. • TYPETPMX with small file - verified 72 position FORMATs worked. • TYPENTSM with test file - verified open-system-style MXG code. • WPS is often updated daily, and as errors were encountered in MXG tests on z/OS and Windows/XP, a new release was generated which did correct each error that could be fixed short-term. The current release is World Programming System 2.3 of May 4.

  7. c. MXG Version 25.11 is required for testing under WPS; its updates eliminate the need for user modifications to MXG Software, and new WPS-specific members (CONFIGW2, MXGWPSV2, JCLINSTW, AUTOEXEW) were created. d. Facilities used by MXG Software not yet in WPS for z/OS: INFILE CCHHR option - needed only for TYPEEREP (EREP,SYS1.LOGREQ) VIEWS - for I/O performance, used in VMXGSUM, which is invoked over 60 time in QABUILDPDB, and also in all ASUMxxxx and TRNDxxxx members. Eliminates a full pass of the input data. PROC CONTENTS - minor, does not report High Block Used size. e. Facilities used by MXG Software not yet in WPS for z/OS and Windows: INFILE with ftp access method - performance slower, more disk space SAS/GRAPH, SAS/STAT, SAS/ETS, etc. Currently WPS supports the Base DATA Step Support and a few PROCs, including PRINT, MEANS, CONTENTS GPLOT, GCHART, with more planned. f. Output differences - minor PROC MEANS printed output - some values printed with one digit more resolution by WPS. - no error in output values spot checked - prevents automated output comparisons

  8. g. Untested - most due to complexity or time to set up multiple inputs: DATA Libraries on TAPE WEEKBLDT, MNTHBLD special TAPE format to DISM, Mod, etc. TESTTRND, TESTANAL - requires extensive test data ANALxxxx - requires test data ASUMxxxx - except ASUMJOBS, ASUM70PR were tested. UTILxxxx - specialized utilities for MXG Tech Support Internal SORT - on z/OS, internal SAS SORT may be required when BY list variables don't fit in first 4096 bytes. Have not confirmed in WPS. NFS Files - not tested. Broken VBS files - not tested.

  9. h. Migration issues - see WPS Migration Instructions: On z/OS, copy all archived SAS Data Libraries on DASD (including HSM migrated) to tape (Sequential) format with the SAS System first. WPS can read SAS Data Libraries in SEQUENTIAL format on tape or DASD. WPS cannot read DASD format SAS Data Libraries i. Customer reports: Several MXG customers have been running their BUILDPDB on z/OS. Early tests had to make source-level-changes to a few MXG members. Most had relatively simple BUILDPDBs with minimal tailoring.

  10. j. MXG Support Position for execution under WPS product: MXG 25.11+ and WPS 2.2 or later are required. Your current MXG Software License Agreement states: Merrill provide continuous product support for MXG Software: When error conditions are the results of errors in MXG code, they will be corrected; error is defined: SAS® execution of MXG code produces either a return code or an ABEND. If you encounter an execution error testing MXG under WPS: You should report the error to WPS Technical Support for initial investigation. If WPS support believes the error is an MXG problem, they can contact MXG, or they can refer you to contact support@mxg.com. If the error can be replicated under the SAS System, it will be corrected per our license terms. For data errors, (INPUT STATEMENT EXCEEDED RECORD etc) Contact MXG Support by sending the FULL job log, and maybe - to ftp the raw data file that caused the error - or to ftp your USERID.SOURCLIB (MXG tailoring) libraries

  11. 2.A. Run time comparisons of SAS and WPS - NOVEMBER 2007: BUILDPDB,ASUMJOBS,ASUM70PR with 448 MB SMF File z/OS Comparison CPU TCB Elapsed ----------- Compressed ---------------- minutes minutes Work Size PDB Size CICSTRAN Size SAS: 3.88 5.20 209 MB 55 MB 59 MB WPS: 10.67 18.50 233 MB 59 MB 67 MB Windows/XP Comparison SAS: 2.18 Note 1 65 MB 63 MB WPS: 2.71 Note 1 78 MB 70 MB Note 1: Neither WPS nor SAS provide a way to track maximum work space on ASCII, except for free space monitoring with DIR

  12. 2.B. Comparison of SAS V9.2 and WPS 2.3 on z/OS. May 2008: BUILDPDB and ASUMs with 448 MegaByte input SMF file; z/OS Step Totals: SAS V9.2 WPS 2.3 RATIO mm:ss sec mm:ss sec Total CPU time 03:45 225 08:30 510 2.26 Total Elapsed time 08:00 480 16:58 1018 2.10 Total EXT Memory 104 MegaBytes 188 MegaBytes Total SYS Memory 11 MegaBytes 504 MegaBytes Total Memory 115 MegaBytes 692 MegaBytes SMF Data Step mm:ss sec mm:ss sec Elapsed time 01:12 72 03:27 207 2.85 SMF Read Rate 6.22 MB per sec 2.16 MB per sec

  13. 3. Revisions to SAS Clones article in MXG Technical Newsletter FIFTY: WPS is no longer vapour-ware. The company had bent over backwards to provide corrections. When originally written in JAVA several years ago, WPS performed so poorly, with run times at least ten times worse than the current release, that JAVA is no longer used in the product. So the offload of your SAS CPU time to a zAAP won’t happen. Items listed in sub-paragraphs i, ii., iii., and iv. have all been addressed to my satisfaction, with the exception of the items that are listed above in this note. Pricing has been significantly reduced from those original IBM prices. As an example, in late 2007, an MXG site in the USA was quoted an IBM price for a 21 Value Unit system (about 1000 MIPS) of $42,000 first year and $8,400 for renewals. WPS can be licensed through IBM or through World Programming; their home page is at http://www.teamwpc.co.uk

  14. 4. Summary: Most of MXG has compiled successfully under WPS on both z/OS and Windows/XP. BUILDPDB has compiled and processed SMF data on both z/OS and Windows/XP. Not all of MXG has been tested with data. WPS is still in development; January’s 2.2 had five errors. This week’s new WPS 2.3 claims to have corrected them. WPS on z/OS requires thrice the CPU and Elapsed Run Time of SAS. It is claimed that WPS 2.3 has improved z/OS performance. WPS on Windows/XP run times are similar to SAS run times. Disk Space required on both platforms are similar. So, it is safe now to test MXG under WPS.

  15. 5. QA Steps successfully compiled and executed with dummy input. STEP 3. CREATE FORMAT LIBRARY STEP 4. RUN TESSNT STEP 5. RUN TESSIBM STEP 6. RUN TESSIBM1 STEP 7. RUN TESSIBM2 STEP 8. RUN TESSIBM3 STEP 9. RUN TESSUSER STEP 10. RUN TESSUSR1 STEP 11. RUN TESSOTHR STEP 12. RUN TYPSCMHM STEP 13. RUN BUILDPD3+ASUMS STEP 14. RUN BUILDPDB+ASUMS STEP 15. RUN TYPERMFV STEP 16. RUN TYPECMFV+TYPEMVCI STEP 17. RUN TESSHSM STEP 18. RUN TESSFACO

  16. STEP 20. RUN TESSVM STEP 21. RUN TESSCRAY STEP 22. RUN TESSHPCS STEP 23. RUN TESSUNIA STEP 24. RUN TESSUNIK STEP 25. RUN TESSQAPM STEP 26. RUN TESSQACS STEP 27. RUN TESSTUX STEP 28. RUN TESSPW STEP 29. RUN ASUMPRTR STEP 30. RUN TYPEIMS7 STEP 31. RUN TESSIMSD STEP 32. RUN COPYSTEP STEP 33. RUN TESSTRND - partially completed, not WPS fault. STEP 34. RUN TESSMNTH STEP 35. RERUN TYPE-S FOR NON PROC SORT FOR DATASET LABEL STEP 36. RUN CROSSREF STEP 37. RUN DOCVER - output DOCVER file is identical.

  17. C. CPU Parked Time Metric. PR/SM data for LCPUADDR 5 in dataset TYPE70PR: "Online Duration" |======================SMF70ONT===========================| 299.97 Online, "Parked" Online, "Dispatched or Not Parked" |=====CPUPATTM======|======= (SMF70ONT-CPUPATTM) =========| 103.22 196.75 Online Online "Dispatched" "Not Parked" |====LCPUPDTM====|======PATWAITM======| 96.80 99.96 MVS data for CPUID 5 in dataset TYPE70: SMF70WAT |=ORIGWAIT=| 0.0000

  18. This data for LCPUADDR=5 shows a CP engine that was parked for 103 seconds of that 5 minute interval. RMF subtracts the SMF70PAT parked duration from the SMF70ONT online duration to calculate the Percent MVS Busy value. In this interval, ORIGWAIT was zero for this engine, as MVS never entered the wait state on that engine, so RMF calculates the MVS busy percent as: PCTMVSBY= 100*(SMF70ONT-ORIGWAIT-SMF70PAT)/(SMF70ONT-SMF70PAT); PCTMVSBY= 100*( 299 -0 -103) /(299 -103) = 100% The IBM calculation of the PCTCPUBY, the LPAR CPU busy percent, is NOT altered by parked time; PCTCPUBY=32%, calculated as PCTCPUBY= 100*(LCPUPDTM/SMF70ONT); = 100 * (96 / 299 ); The "PATWAITM", the time when the CP engine is "not parked", is the time when this CP engine could/should have been parked, but was still online and not-dispatched, because the algorithm to park a CPU only executes occasionally. It is not created in TYPE70PR.

  19. D. zIIP and zAAP measurements when they are faster than CPU engines. When specialty engines are faster than the speed of your CPs, there is a normalization factor to convert the recorded seconds to their NORMALIZED (EQUIVALENT) time, as if they had executed on your CPs. In all MXG workload datasets, TYPE72GO and RMFINTRV, (and TYPE30), all time variables for zIIPs and zAAPS are NORMALIZED seconds, and all of the service units are segregated by engine type. However, the IBM RMF reports present these data quite differently. This system has the normalization factor, R723NFFS=569/256=2.222, that is, one second of zIIP is equal to 2.222 seconds of CP time. ================================================================ MXG Dataset TYPE72GO dataset values: ================================================================ SERVICE CPUUNITS ZIPUNITS CPUTCBTM CPUZIPTM 3,932,091 1,793,920 2,137,167 178.92 213.16

  20. RMF WORKLOAD REPORT: Under "SERVICE TIMES", the RMF "CPU" value of 392.9 seconds is the total of the real CPU time on CP engines, plus the NORMALIZED CPU time on the zIIP and zAAP engines; it is NOT the CPU "TCB" time. ( 392.9 = 178.92 + 213.16 "RMF CPU" = CPUTCBTM + CPUZIPTM ) But also under "SERVICE TIMES", the RMF "IIP" (zIIP) value of 96.1 seconds is the UN-NORMALIZED, raw, seconds on the zIIP engine. And the RMF "AAP" value for zAAPs is also the UN-NORMALIZED value. And under "SERVICE", the RMF "CPU" value of 3931K is the TOTAL SERVICE units from CPs, zIIPs, and zAAPs. REPORT BY: POLICY=OWL WORKLOAD=CSSDDF TRANSACTIONS ---SERVICE---- SERVICE TIMES ---APPL %--- AVG 0.23 IOC 0 CPU 392.9 CP 4.98 MPL 0.23 CPU 3931K SRB 0.0 AAPCP 0.00 ENDED 51 MSO 0 RCT 0.0 IIPCP 0.07 END/S 0.01 SRB 0 IIT 0.0 #SWAPS 0 TOT 3931K HST 0.0 AAP N/A EXCTD 0 /SEC 1092 AAP N/A IIP 2.67 AVG ENC 0.23 IIP 96.1

  21. While the workload datasets have normalized CPU time, in all of the "hardware" datasets, TYPE70, TYPE70PR, ASUM70PR etc., the CPU times for the zIIP and zAAP engines are the raw seconds of CPU Dispatch Time on those engines, and is NOT normalized. As a result, then, the total ZIPACTTM recorded in TYPE70 for the above system for the day was 10,887 seconds, but the total CPUZIPTM in TYPE72GO for the day was 23,079 seconds. Those 10,887 raw hardware seconds would be 24,190 normalized seconds so the zIIP capture ratio at this site is 23079/24190 = 95.4%.

  22. E. The CPU cost of performance monitoring and capacity planning. One MXG user reports they currently write 500 GB of SMF data per day or an average rate of 6 MegaBytes per second across all platforms. They dump SMF multiple times each day, and build multiple "PDB's" throughout the day, and run many ad hoc analysis reports as well. They have SMF, RMF, OMEGAMON, and NETVIEW monitors consuming CPU. The daily total CPUTM for each of their workloads were: OMEGAMON 28:56:37 MXG JOBS 19:05:01 RMF III 12:20:05 RMF I 6:29:11 SMF DUMPS 4:12:30 MONITORS 2:17:10 SMF ASID 0:29:16 TOTAL CPUTM 73:30:50 = 2% of 3744 HOURS with 156 CPs

  23. Thus this sites total daily cost of 74 CPU hours is an average use of 3 CP engines all day long, but with 156 CP Engines, that is ONLY 2% of the installed CP engine capacity, for the entire CPU cost of performance monitoring, data collection, building PDBs, archiving, and all MXG daily reporting and ad hoc analysis. to read 518GB of SMF data 838K TMC records 902K DASD DSN records (DCOLLECT) 27K DASD VOL records (DCOLLECT) The following graph shows the hourly CPU consumption of these workloads.

More Related