SRS Secure Desktop Project – Running Without Administrator Privileges

1. SRS Secure Desktop Project – Running Without Administrator Privileges Barry Hudson Desktop Systems Team Lead SRNS Aiken, SC 29808 barry.hudson@srs.gov 803/725-8463

2. Abstract • SRS has approximately 10,000 PCs, the majority of which are centrally managed. The major project for 2008 was to “secure” these desktops by removing routine user of Administrator accounts. This was completed in parallel with a project to employ all applicable FDCC Group Policies. • In less than 8 months, all managed desktops were converted to the “Secure Desktop” model. Over 9000 XP systems were converted to NTFS, administrator privileges removed, and software called BeyondTrust Privilege Manager was used to elevate privileges when needed for routine operation and software updates. • This presentation will outline the challenges and solutions used to make the transition to “Secure Desktop” without making it “Desktop Lockdown”. It was achieved ahead of schedule, with existing staffing levels, and with fewer than 100 visits to the desktops. SRS Secure Desktop Project – Running Without Administrator Privileges

3. Drivers and Background • PC configuration management has been a consistent OIO audit finding since 2001 • Various policies, procedures, and processes have been implemented over time but the administrative privileges have not been removed from the user • Another Audit was scheduled for September 2008 SRS Secure Desktop Project – Running Without Administrator Privileges

4. This Is Just 1 Piece of The Puzzle • Overview of Comprehensive Desktop Management at SRS • WSUS patching, WinInstall updates, Symantec AntiVirus • Cisco Clean Access Posture Check and Remediation • IVIS scanning • Vulnerability and Patch Management Team (VPMT) SRS Secure Desktop Project – Running Without Administrator Privileges

5. VPMT Weekly Meeting to review, status & track High risk vuln. to closure within 45 days or exile from network Daily review by VPM Rep and remedy of LHF within 24 hrs or exile from network Manual Internal Netstat (Network Statistics tool) Supplemented with Selective External NMAP (Network Mapper open source network security Audit Tool including identifying services offered) scans on a weekly basis Automated Daily Sans Top 20 Scans via Hercules and IVIS V&PM Nikto / Web Application scans to Capture vulnerabilities on a monthly basis (Nikto is open source web server scanner performing multiple checks Oracle Oscanner Scans on a quarterly basis List of devices Requiring manual vuln. & web application scans* Alter routine scans to incorporate port info found IVIS (Nessus Based Engine) IVIS VPM Reports Delayed scan request run within 30 days? IVIS Low Hanging Fruit / Easily Exploitable Vulnerabilities Report Data No Data Routine Scans Follow Up Scans Unseen Host Scans Ad Hoc Scans Yes Quarterly Scan Results If IVIS scans show vulnerabilities, follow up scans occur daily. Every hour devices on the network are checked for record of scan in last 7 days; if not full scan commences. Scan lists are created from ARP Table such that the entire site is covered within 1 week. Cisco Ops Ware/ NAS Policy Compliance Check run by Networks And reported monthly. Nessus Scans Secondary Network Secondary Network (Stand Alone) Secondary Network SRSNet Secondary Network HP Jet Admin Scanning Function run Daily for discovery. Secondary Network SRS Secure Desktop Project – Running Without Administrator Privileges *Ad hoc scans are requested with a VPMT rep

6. Project Scope • Remove “routine use” of users with Administrator privileges • Limits Malware propagation • Users would be limited to install approved, standard applications (i.e. WinInstall Applications). • Restricts implementation of local peripherals. • Security configuration management is strictly enforced • Users less able to install vulnerable software • Group Policy enforcements where possible • Implement FDCC policies • Work in parallel with other scanning, patching, and security initiatives • Finish quickly, in time for the “Going 4 Green” audit SRS Secure Desktop Project – Running Without Administrator Privileges

7. What PC’s Are Out There ? • Operating Systems • 7000 Windows XP • 2000 Windows 2000 (Reload with XP) • 1500 Controllers and specialty systems • “Owner” is Admin, others have non-Admin access • Software Inventory Results • Almost 200 Centrally-managed Applications (WinInstall) • Over 40,000 identified self-installed applications • Will they continue to run? • What happens if they need to be reinstalled or updated? • About 2500 systems had no additional software SRS Secure Desktop Project – Running Without Administrator Privileges

8. But Wait! Removing Administrator Rights Doesn’t Buy You Much • Programs can be installed in the user space • Current User registry is not protected from Run, RunOnce, etc. • Why not whitelist all approved applications • And scan more often • And it will cost $500,000 or more to implement • Our time would be better spent doing more of what works SRS Secure Desktop Project – Running Without Administrator Privileges

9. But Wait! Removing Administrator Rights Doesn’t Buy You Much • Programs can be installed in the user space • Current User registry is not protected from Run, RunOnce, etc. • Why not whitelist all approved applications • And scan more often • And it will cost $500,000 or more to implement • Our time would be better spent doing more of what works BUSTED Hijacked processes run with the rights of the logged-on user SRS Secure Desktop Project – Running Without Administrator Privileges

10. Orders From Headquarters SRS Secure Desktop Project – Running Without Administrator Privileges

11. Impacts to the Business & Status Quo • Users will not be able to perform activities that require administrative access. • Installing software from CD’s • Creating file shares • Adding software drivers such as scanners and printers • Certain other system modifications • Activities requiring administrative privileges must be performed by IT support personnel or special accounts for the users • Existing supported applications will be assessed and modified to install and run in this environment • User supplied applications will be accommodated or converted to managed applications SRS Secure Desktop Project – Running Without Administrator Privileges

12. Planning Assumptions • Barry, you’re no longer in the customer service business, you are in the security business • Things will break • Processes will fail, but not always immediately • We will learn as we go • Some systems will be easier to migrate than others • Focus on Managed Desktops first (8500) - XP Only • Review of WinInstall applications • Review of local applications • Pick the “least likely to fail” systems first • Pilot migration of some “tough” systems • Then tackle controllers and shared systems (1000) • And finally specialty systems (500) • And hope everything runs at FY year-end closure SRS Secure Desktop Project – Running Without Administrator Privileges

13. Design Assumptions • The site needs to do business in the manner they are accustomed to • Proactive planning will establish working footprint but likely anticipate only <80% of the issues • Costs: Software, Staffing, Lost Productivity • $500,000 + 10 FTE + TBD > $1.5 million • Increased support staff, apps review, new processes • Not enough time to test all standard apps will load and run • Things will break, Processes will fail, We will learn as we go • 40,000 apps that we have not idea how to test • We will allow deferrals (the thorn in my side) • Doing FDCC and Secure Desktop at the same time • Makes it hard to determine “what broke it” SRS Secure Desktop Project – Running Without Administrator Privileges

14. Staffing requirements • What we asked for: • Desktop Team: 2 people fulltime for 1 year • Field Support: 2 visits per year x 1 hour x 10,000 systems = 20,000 hours = 10 FTE’s • What we got: • 2 Help Desk Agents • Desktop Team delayed priorities for 6 months • An accelerated schedule (Get the pain over quicker) SRS Secure Desktop Project – Running Without Administrator Privileges

15. Selling It • Tell them why, when, and how • Pick a non-threatening name • Secure Desktop vs. Desktop Lockdown • Publicity campaign, Sitewide Emails, Roadshows to Customers • Involve customers, Computer Security, IT, and management • Weekly meetings of 20+ stakeholders • 100+ issues and concerns • IT and DOE Security goes first (walk the talk) • Provide a safety-valve (add the user back as Administrator) • Things will break, Processes will fail, We will learn as we go • I made a Promise • If you can’t still do your job with Secure Desktop, that means I have not done my job right. SRS Secure Desktop Project – Running Without Administrator Privileges

16. Publicity Campaign SRS Secure Desktop Project – Running Without Administrator Privileges

17. Early Discoveries • Life as Non-Admin (life changes) • Restricts access to registry, printer installs, software installs • Can’t setup scheduled tasks • Life as Non-Admin with NTFS (life gets really interesting) • NTFS Restricts access to files and folders (eg c:\Windows, c:\Program Files, Local Apps, default profiles, and more) • Can’t read other users’ or All Users profiles • Some apps might need to be modified or sections of PC opened up for them to run SRS Secure Desktop Project – Running Without Administrator Privileges

18. Technical Tools to Make It Work • BeyondTrust Privilege Manager to elevate privileges when needed • Approx 160 rules in place • To install software • To run some software • Elevate System processes (eg TCP/IP Configure, Add Local Printer, VPN Firewall, Plug and Play) • Provides Inheritance so that auto-installers can retain rights • Use CACLs to tighten or loosen file and folder permissions • Approx 100 exceptions needed • Some apps write INI files to c:\program files • Refine Registry permissions (CACL or BeyondTrust) • Some apps change HKLocalMachine or HKClassesRoot • Add and manage Non-person domain account with privs • For hands-on support • For system updates and automated processes SRS Secure Desktop Project – Running Without Administrator Privileges

19. More About BeyondTrust • Attach permission levels to Windows applications and processes • Integrated with Active Directory and applied through Group Policy • Policy is applied by creating rules in the Group Policy Object Editor (using their GUI) • Operates transparently to the end-user • Permits “least privilege” elevation • Costs about $30 per seat ($300,000 plus 15%) • Container license vs. Domain license • Computer object limit and user limit SRS Secure Desktop Project – Running Without Administrator Privileges

20. Examples of Rules • Rules can permit or elevate based on • GUID or URL-specific ActiveX controls • Residence in a particular Folder • Hash of the file • MSI that is being installed • Specific Path of the file • Other attributes • Recommendations: • Use a Hash when possible • Multiple versions (eg Flash4 and Flash5) are allowed • Avoid Path and Folder rules if you do not control the fileshare • Don’t open a path or share where anyone can drop an installer or EXE • Look for inadvertent inheritance to downward processes • An elevated DOS box can be a big hole SRS Secure Desktop Project – Running Without Administrator Privileges

21. BeyondTrust Configuration Screen SRS Secure Desktop Project – Running Without Administrator Privileges

22. Process Tools to Make It Work • How to permit users and field support to regain Administrator rights when needed • Locally written tool to add Admin back to PC and log the reason • Temporary Restoration of Administrator Rights (TRAP) • 20-30 per day added by Help Desk • TRAP process should be followed by a scan after user does install • 200+ “Admin Restores” exist at any point in time • The PA process and PC-SPPT-xx groups • PA – Personal Admin account allows selected users to support their PC • PC-SPPT-xx groups added to specific Workgroup PCs (shadow support) • The CS/DA process • Computer Support accounts for installers and Field Support • Desktop Admin accounts for my staff • RunAs and Remote Administration • Non-secured machines that are offline are identified and secured within 2 days of connection • Daily inventory to check settings, TRAP abuse, lost sheep returning SRS Secure Desktop Project – Running Without Administrator Privileges

23. Who Gets to Go First? • Management champions • IT and DOE Security goes first (walk the talk) • SRNL gets a gold star (150 of the 1st volunteers) • Ask for volunteers (motivate them with “get better help before the storm”) • Verify laptop, VPN, and off-line operations • Email campaign with “magic button” to migrate now • Sent to users with no known extra applications (Dear User:) • Convincing users to participate. Not everyone is “special” • Look, SRNL did it and is still running! • Allow Deferrals only for Good Reason (preferably classes of systems, eg Doc Mgt, Maintenance, Controllers) • Publicize your success, acknowledge your weaknesses SRS Secure Desktop Project – Running Without Administrator Privileges

24. The Planned Schedule 10-12/07 proof of concept 1/08-3/08 100 user pilot 30 days to regroup 4/08-5/08 1000 “easy” systems 6/08-12/08 6000 total migrated The Forced Schedule 10-12/07 proof of concept 1/08-2/20/08 50 user pilot 2/25/08 500 users added 2/26-3/5/08 1500 “easy” systems 3/08-5/08 6000 total migrated 6/08-12/08 deferrals 1/09-2/09 who is hiding? The Schedule SRS Secure Desktop Project – Running Without Administrator Privileges

25. Migration Rate Don’t run out of licenses! “Push it till it breaks” Deferrals released “Push it till it breaks” The 2000 “jump-start” provides confidence. Sure, they are still running but what about at the end of the month? SRS Secure Desktop Project – Running Without Administrator Privileges

26. Automating the Migration Process • Make all of WinInstall “BeyondTrust Aware” • NTFS Conversion via email Magic Button invitations and forced WinInstall • Pre-req before getting added to the Scheduler list • The Secure Desktop Scheduler • Triggers at login if you are on the list, NTFS, and an Administrator • Launches the upgrade process in WinInstall • One giant WinInstall • Install BeyondTrust Privilege Manager • Install Basic CACLS to secure additional selected folders (approx 10) • Add Local Printer shortcut • Temporary folder for 16 bit applications in location not under c:\windows • Remove Administrators from Computer and randomize local password • Move Computer to BeyondTrust Container • Apply Common CACLS to open additional selected subfolders (approx 100) • Write “all done” logfile • About 100 systems did not migrate and required reload SRS Secure Desktop Project – Running Without Administrator Privileges

27. “What Broke It” and How to Remedy It • Secure Desktop gets blamed for everything! • Diagnosis • App won’t install or won’t run? • Is everything broken or something specific? • Is BeyondTrust running? Do you have the current rules? • Add user back as Admin and see if it fixes it • This is your safety net • Keeps the business running while you figure it out • Look for activity in the NTFS-protected folders (Program Files, Windows, System32, etc) • Look for activity in the Registry • Triage to identify commonalities • Repair • Elevate the program (hash vs. path) • Liberalize rights on sub-folders or files (CACLS) • Change program configuration (set INI file or prefs files to write elsewhere) SRS Secure Desktop Project – Running Without Administrator Privileges

28. Phase-in of New Secure Desktops • Unfair to put installers on the front lines • Deliver and add user as Admin • Permit users or installer to add software • Scan the system for vulnerabilities • Add to the SD migration list after 3 days • All Windows 2000 were re-built on-site as Secured • Deliver “As-Secure” at the end of the project • Local Admin used only to add to domain • Then remove all Admins SRS Secure Desktop Project – Running Without Administrator Privileges

29. Unexpected Issues • Chicken and Egg situations • Have to be an Admin to become secured • But our goal is to eliminate Admin users • How to pre-build a secured machine • Dealing with the absence of a universal Administrator account • There is no local administrator to “break in” with • 90 day lost of trust issues • Cached login with last good user • Set Owner Issues • Some files are owned by the installer and cannot be accessed by others or have provided unwanted access to install folders • Essential things did not work and need elevation • Defrag, Clock, Ipconfig, RunOnce after installs • Issues with multi-user systems • No unrestricted place to put “turnover” files • Screensaver locked at shift change SRS Secure Desktop Project – Running Without Administrator Privileges

30. Ongoing Maintenance • Daily Un-TRAP of Admin Restores • Look for abuse • Propagate PA and PC-SPPT-xx accounts • Verify new installs are secured • All scanning and remdiation activities must be Secure-Desktop aware • Add rules as issues arise (about 4 per month) • New products • Stuff breaks • Updates to existing rules • BeyondTrust product enhancements SRS Secure Desktop Project – Running Without Administrator Privileges

31. Summary • Project success despite objections from users and reluctance of IT staff • Early 500/day was a crazy idea but provided valuable insight and confidence • Almost finished before we had planned to get started • Critical success factors • Publicity campaign • Top level management support • Acknowledgement that things would break • Availability of a relief valve (Restore Admin user) • Ability to select and throttle the update list SRS Secure Desktop Project – Running Without Administrator Privileges

32. Questions Barry Hudson Desktop Systems Team Lead SRNS Bldg 773-51A Aiken, SC 29808 barry.hudson@srs.gov 803/725-8463 SRS Secure Desktop Project – Running Without Administrator Privileges