1 / 13

Uncertainty in Automation: Anomaly Detection in Event-Based Systems

Uncertainty in Automation: Anomaly Detection in Event-Based Systems. Dawn Tilbury Linday Allen (PhD) and John Broderick University of Michigan. Outline. Example problem: inconsistent logical behavior Developed solution: Anomaly detection Model generation using observed data

mireya
Télécharger la présentation

Uncertainty in Automation: Anomaly Detection in Event-Based Systems

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Uncertainty in Automation:Anomaly Detection in Event-Based Systems Dawn Tilbury Linday Allen (PhD) and John Broderick University of Michigan

  2. Outline • Example problem: inconsistent logical behavior • Developed solution: Anomaly detection • Model generation using observed data • Performance assessment of models using known “good” and “bad” behavior • On-line anomaly detection • Industrial Application • Academic assumptions meet industry realities • Resolution and results • Lessons learned

  3. OPC Tag OPC Tag Response Response s1 Example from our testbed Correct, typical behavior Incorrect behavior PartReady Part1 Release Pallet PartReady Part1 LoadPart1 • No model of entire system’s correct behavior • Manual inspection required to find this anomaly • Laborious, offline

  4. Events All Events Op. Feedback Anomalies Approach: Anomaly detection using model generation Plant Plant • Goal: diagnosis of system level event-based faults in mfg systems without use of pre-existing formal model Anomaly Fault diagnosis detection Controller Controller Logic Control System Operator • Method: • Generate models based on training data • Detect anomalies on-line by comparing traces to models • Advise the operator when anomaly occurs

  5. Knowns & Unknowns • Known: • Resources in system • Robots • CNC machines • Pallets • Measurable: • OPC tag changes • Communication events between controllers • Unknown: • Formal model of the systemCould be constructed but is time-consuming and error prone • Logic control codeWritten by different people at different times in different languages • Correct event orderMany different orders may be acceptable

  6. Resource Info ok efad... Model Generation ok abfc... adbe... ok Solution approach Given resource information and strings of “ok” events Given a new string, determine whether the models accept it (weight by model performance) Given some “ok” and “not ok” strings, compute the performance of each model abfe... 6 31 22 ok Performance Assessment abfe... Fault Detection OR Create a set of models that can generate these strings abfe... not ok eafd... not acea... If not, where is the anomaly

  7. Anomaly Detection Method • Inputs: • Streams of events from system in operation • Resource information, including mapping of events to resources • Outputs: • Set of models that represent the system behavior • Model performance on training data • On-line detection: • Score for each string – anomalous or not

  8. G2 G1 M1 M6 2 3 4 1 Machining Cell: Physical Set-Up • Problem: G2 will have raw parts and at least one CNC available, but G2 incorrectly waits • Resources: • Gantries, CNCs, buffer at hand-off • Events: PLC data recorded via Ford data collection system Entry Hand-off Exit Reject

  9. Driving logic Data collection set-up • Data from each machine & gantry • Bits include: Cycle End, Good/Bad Cycle, Wait Aux, Blocked, and Starved • PLC message generated each time particular bit changes occur • Approx. 11,000 parts worth of data IT System FunctionBlock FunctionBlock DrivingLogic Driving Logic PLC PLC (270,000 PLC messages)

  10. Identified Inconsistencies • What we thought we would get: • Well-defined strings of events • Events that acquire/release resources recorded • Unique mapping of PLC bits to events • Many strings, starting from the initial state, labeled as “good” or “bad • What we got: • Not every event triggers a message  multi-bit change (order is uncertain) • Not all resource events captured in data collection • Some bits used for multiple purposes • One huge log file with no defined “beginning”

  11. Resolution of Inconsistencies

  12. Word 18 bits 8-10 give the CNC ID Ford data Gantry waiting: word 19 bit 9 is high

  13. Lessons learned • Sometimes you can adapt/improve your method to handle given uncertainties • Multiple models when system model unknown • Multiple bit changes  uncertain event order • Initial state unknown • Advise operator instead of closing the loop • Sometimes you have to decrease the uncertainty by improving the system • Consistent bit/event mapping • Unobservable events for acquiring resources

More Related