Hernán J. Larrea Netbackup 6.5 Backupprocess
Disclaimer • Symantec Corporation has notparticipated in thedevelopment of thisdocument. • Allsourceswillbeproperlymentioned at theend of thedocument.
Abouttheauthor I’m Hernán J. Larrea, an IT engineer with 8 years of experience in the field. I decided to address my career in Wintel technologies, specially Active Directory, Backup and Recovery environments, Storage solutions and a little programming in .NET. I’ve compiled this document since I believe it is very important for Netbackup administrators to count with this knowledge at the time of troubleshooting, Symantec has good articles and documents describing this processes, but compiling this in form of slides and animations helps to concentrate the eyes and the mind, in different and specific matters making learning much easier. If you want to know more about me, I invite you to visit my personal website http://www.hernanjlarrea.com.ar or my blog at http://blog.hernanjlarrea.com.arand if you want to email me, you can do it at firstname.lastname@example.org. Hernán J. Larrea
Startupprocess • When the NetBackup master server boots up, a script automatically starts all services, daemons, and programs required by NetBackup. • The same is true on a media server: NetBackup automatically starts additional programs as required, including robotic daemons.
Startupprocess 2 • A daemon that executes on all servers and clients is the NetBackup client daemon, bpcd. On UNIX clients, inetd starts bpcd automatically so no special actions are required. On Windows clients, bpinetd performs the same functions as inetd. • Note that all NetBackup processes can be started manually by running the following: /usr/openv/netbackup/bin/bp.start_all
Backupprocesses (Job Scheduling) Starting with NetBackup 6.0, the scheduler process bpsched is replaced by the following new services: • nbpem service (Policy Execution Manager): creates a persistent job worklist, starts each job when it is due (no wakeup interval), and sets a timer for the next due job. • nbjmservice (Job Manager): accepts requests from nbpem to run backup jobs, or to run media jobs from commands such as bplabel and tpreq. nbjm acquires resources for each job, such as storage unit, drives, media, and client and policy resources, and executes the job. • nbrb service (Resource Broker): allocates resources in response to requests from nbjm. nbrb acquires physical resources from nbemm (the Enterprise Media Manager service), and manages logical resources such as multiplex groups, maximum jobs per client, and maximum jobs per policy. nbrb is also responsible for initiating drive unloads, and manages pending request queues.
EMM Server and master server The NetBackup master server and the Enterprise Media Manager (EMM) server can be on the same physical host or on different hosts. • Master server: responsible for running jobs as configured in NetBackup policies. The nbpem and nbjm services run only on the master server. • EMM server: allocates resources for one or more master servers. The EMM server is the repository for all device configuration information. The nbemm and nbrb services run only on the EMM server. • nbemm service: centralizes resource selection. nbemm maintains devices, media, and storage units in a relational database. Prior to NetBackup 6.0, resource selection was handled by bpsched, bptm, ltid, and the robotic device daemons.
Backupprocess As you might already know, Netbackup operation implies communication across many, many daemons and processes, here you will find what happens when a backup starts.
Beforebegining… • Eventhetextspeaksabout tape and disk storageunits, diagramswillonly describe theprocessfor tape (theprocessisonlydifferentonly in onestep, 15). • Eventhetext describes thisis a backupon a UNIX host, processisjustthesamefor Windows machines.
Backupprocess Afterboot up: 1) A start-up script launches bprd on the master server and ltid on the master server and all media servers. All other daemons and programs are started as necessary, including nbpem, nbjm, nbrb, and nbemm. EMM & Master Server bprd 1 nbpem Media Server Client 14 ltid
Backupprocess The policy execution manager service (nbpem) does the following: 2 )Gets the policy list from bpdbm. 3) Builds a work list of all scheduled jobs. 4 )Computes the due time for each job. 5 )Sorts the work list in order of due time. 6 ) Submits to nbjm all jobs that are currently due. 7 ) Sets a wakeup timer for the next due job. 8 ) When the job finishes, re-computes the due time of the next job and repeats at step 5. EMM & Master Server bprd 1 2 2 bpdbm nbpem 6 nbjm Media Server Client 14 ltid
Backupprocess Next, the job manager service (nbjm) does the following: 9) Issues a single request (with a request ID) to nbrb, for all resources required by a job. nbrb gets the storage unit, tape drive, and media id information from nbemm and allocates client and policy resources. nbrb returns to nbjm an allocation sequence containing one allocation for each resource (each allocation contains a unique ID). nbrb also returns allocation data for the specific resource type. nbrb also returns the request ID along with the allocations so that nbjm can correlate the response with the right request (and job). 10) nbjm starts the backup by using the client daemon bpcd to start the backup and restore manager bpbrm. EMM & Master Server bprd 1 2 2 bpdbm nbpem 6 nbjm 9 9 9 nbemm nbrb 10 Media Server Client bpcd 10 bpbrm 14 ltid
Backupprocess Next, the backup and restore manager (bpbrm) does the following: 11) Startsbptm. 12) Starts the actual backup (or archive) by using the client daemon bpcd to start the backup and archive program bpbkar on the client. EMM & Master Server bprd 1 2 2 bpdbm nbpem 6 nbjm 9 9 9 nbemm nbrb 10 Media Server Client bpcd bpcd 10 12 12 11 bptm bpbrm bpbkar 14 ltid
Backupprocess Next, the backup and archive manager (bpbkar) does the following: 13) Sends information about files within the image to bpbrm, which directs the file information to the NetBackup file database. The information is sent by means of bpdbm on the master server. 14) Transmits the backup image to bptm. This is accomplished in one of two ways : If the media server is backing up itself, bpbkar stores the image block-by-block in shared memory on the media server. If the media server is backing up a client on a different host, the bptm process on the server creates a child process of itself. The child receives the image from the client by means of socket communications and then stores the image block-by-block in shared memory on the server. EMM & Master Server bprd 1 2 2 bpdbm nbpem 6 nbjm 9 9 9 nbemm nbrb 10 Media Server Client bpcd bpcd 10 12 12 11 bptm bpbrm bpbkar 13 14 ltid
Backupprocess Next, the backup manager for tape or disk does the following: 15) The bptm or bpdm process on the server takes the image from shared memory and directs it to the storage media. ■ If the storage media is tape, bptm requests information for the first media and drive to use, by exchanging information with nbjm. bptm sends mount requests for specific media and drives to the NetBackup Device Manager (ltid), which causes the media to be mounted on the appropriate devices. If, during the backup, a tape span is required, bptm again exchanges information with nbjm to release the correct tape and to get another one. nbjm exchanges information with nbrbto accomplish this. ■ For SharedDisk, AdvancedDisk, and OpenStorage: bptm requests the volume from nbjm, which passes the request to nbemm to choose the volume and media server to use. nbemmcallsnbrmms on the media server that was chosen to mount the volume. If a span is required, the same steps are used to mount the volume as described in the previous bullet. ■ For BasicDisk: bpdm writes the images to the path configured in the disk storage unit. The system disk manager controls the actual writing of data. In the case of an archive, bpbrm deletes the files from the client disk after the files have been successfully backed up. EMM & Master Server bprd 1 2 2 bpdbm nbpem 6 nbjm 9 9 9 nbemm nbrb 10 Media Server Client bpcd bpcd 10 12 12 11 bptm bpbrm bpbkar 13 15 14 ltid
Backupprocess Next, the job manager service (nbjm) does the following: 16) Receives completion status of the job from bpbrm. 17) Releases resources to nbrb and returns status to nbpem. EMM & Master Server bprd 1 2 2 bpdbm nbpem 17 6 nbjm 9 17 9 9 nbemm nbrb 10 16 Media Server Client bpcd bpcd 10 12 12 11 bptm bpbrm bpbkar 13 15 14 ltid
Sources • Veritas NetBackup™ Troubleshooting Guide UNIX, Windows, Linux Release 6.5