What to do when you are the only one in step

1. What to do when you are the only one in step Dr. M. Smith, S. M. I. L. E. Hardware / Software Co-design Laboratory, Dept. of Electrical and Computer Engineering, Dept. of Radiology, University of Calgary

2. Talk Overview • Reason for doing the research • Brief discussion of what “everybody else was doing”. • Description of the “little project we planned to do” • Our simulation study and all the problems that arose. • Why so many problems? • What we are currently doing (to solve the issue).

3. Story 1What you tell everybody else. • Start of World War II with many men conscripted and being readied to be sent over-seas. • After basic training, the men parade through the town (in front of their kin-folk) prior to embarking on a train. • Mother (wife) and son watch the parade. • Son – wanting to believe in the perfection of his father • “Look, Mother! Father is the only one in step.

4. Story 2 -- Have confidence in yourself and your research cability • Sign on my desk • given to me by one of my graduate students • It’s difficult being perfect Buts somebody’s got to do it!

5. Background HEMORRHAGIC ISCHEMIC • Stroke --the third leading cause of death and the leading cause of adult disability. • Goal of therapeutic strategies is to minimize the progression of tissue damage in the acute phase of the disease. • Methods to rapidly assess acute stroke in individual patients are highly desirable. • 85% of the stroke cases are ischemic strokes due to a reduction of the blood supply by the presence of a clot in a feeding artery (adapted from www.lanacion.com).

6. Methods to measure Cerebral Blood Flow were known (1996) • Track a bolus of magnetic material through the brain (arterial and tissue signals) • Convert changes in “ MR signal intensity” to “concentration curves” using the “magic” log. Formula The technology of any sufficiently advanced civilization looks like magic. – Arthur C. Clarke

7. What “everybody else” was doing • Need to deconvolve “tissue signal” ( cVOI(t) ) by “arterial signal” ( cAIF(t) ) to get “residue function” ( R(t) ). • Peak of residue function provides estimate of blood flow (CBF)

8. Clinical results: Appear to make perfect sense (Calamente, MRM, 2000) • Impact of delay • Impact of Dispersion • a: CBF map. • b: Signal intensity time • A clear delay of 2 sec in the arrival of the bolus can be seen in the right side. • The presence of such delay (and possibly dispersion) introduced a significant underestimation in the CBF map. • The measured right to left ratio in the CBF map is 0.55 due to delay

9. What we were planning to tackle.Signal loss through noise filtering IMPACT OF NOISE FILTERING – LOSS OF SIGNAL

10. Where did the signal loss come from? • Deconvolution causes an enhancement of high frequency noise components. • To stabilize the algorithm, you must apply a filter to reduce the noise. • However, the noise filter also reduces the high frequency signal components – so maximum of residue function is reduced – CBF appears smaller TIME:AMPLITUDELOSS HIGHFREQUENCYLOSS

11. Plan of action“Quick one term project” • Step 1 - “Stand on the shoulders of giants”Repeat what everybody else is doing so we can check we “understand” the problem. • Generate some artificial data (tissue and AIF) • Add some noise • Do deconvolution (standard approach) to get residue function. • Noise filtering removes “high frequency components • Measure CBF as a function of delay / dispersion and tissue type

12. New idea – based on a previously successful MRI reconstruction approach • Generate some artificial data (tissue and AIF) • Add some noise • Do deconvolution (standard approach) to get residue function. • Noise filtering removes “high frequency components • MODEL the low frequency signal components and extrapolate those signals into “high” frequencies • Compare “our CBF” to “their CBF”

13. ARMA modeling – TERA algorithm • Use known low frequency data to generate high frequency data

14. Issue 1 – Insufficient information about how to construct signals • Mathematical formula for constructing arterial signal is given • “Nothing” about how to construct “tissue signal” – we suspect that “either we are missing something obvious (out-of step)” or else construction done by “numerical convolution” rather than algebraic. • “Nothing” specific about how to add noise to get “realistic data”, although some people mention adding “gaussian white noise” to the concentration • Every body discusses low and high “signal to noise ratio” – but nobody says how to measure it.

15. Start putting on the “engineers hat” • Generating data by “convolution” is a delicate process. • If the data is not sampled “fast enough” then “Nyquist” is not satisfied. • MR DSC data sampled at 2.25 seconds • If Nyquist not satisfied then “data” gets distorted at high frequencies (aliasing). • All CBF results “are wrong”, but by “how much” and “when”?

16. Other “engineering stuff”You can get better results “doing it wrong” • Would “everybody else” not doing things the proper “engineer way” impact on our “new” method done the “correct way”?

17. 2 -- We don’t understand the properties of “SVD” (time domain deconvolution) • Need to deconvolve “tissue signal” ( cVOI(t) ) by “arterial signal” ( cAIF(t) ) to get “residue function”. • Peak of arterial signal provides estimate of blood flow (CBF)

18. Use “engineering principles again” • We would expect that frequency domain deconvolution to give same results as time domain deconvolution – except for fine detail • HOWEVER literature is saying “MUCH BETTER RESULTS” are being obtained with SVD than with FT – does not make engineering sense – unless “something wonderful is happening”

19. Problem 2 -- Noise modeling is being done wrong • The MR signal (upper picture) has “gaussian noise” on it (unless very small in intensity and then the noise characteristics change) • This means that adding noise to the concentration curves does not model “clinical data” Added noise Calculated noise

20. Paper 1 – Discussing SNR issues based on true noise model • True SNR of concentration signal changes with MR signal intensity – specific “best” conditions

21. Paper 1 – Discussing SNR issues based on true noise model • Consequences – we believe that everybody is “setting the image parameters” the wrong way

22. Paper 1 • Did not cause much “controversy” • Other researchers have now demonstrated that our predictions are to be found in practice. • Optimize SNR through TE changes and have different MR sequence for tissue and AIF signals • Largely ignored • Difficult to get the “correct” imaging parameters. • Takes too long to get “an DSC image sequence” • “Tissue” signal have low intensity, therefore people “push arterial signals” into an unsatisfactory “high intensity” region to compensate.

23. Next step -- Deconvolution • We have the noise simulation problems understood • Lets try using frequency domain deconvolution (about which we have much knowledge) rather than SVD – time domain deconvolution • As engineers we expect Equivalent results between SVD and FT

24. Trouble is – the FT and SVD answers are very different • FT shows “no time delay effects” that are so evident with SVD. We are really out of step SVD deconvolution FT deconvolution

25. Noise characteristics of SVD and FT differ in unexpected way • Noise Enhancement during deconvolution SVD deconvolution eigen-value thresholding causes “band pass” filtering

26. We have big problems • The delay sensitivity of SVD deconvolution is “breaking” the deconvolution rules • BUT the SVD is a VERY well-known algorithm and NOBODY has reported problems like this in 50 years • The noise effect shows that the SVD filtering is a series of band pass filters. • Band pass characteristics controlled by “eigenvalues” which are identical to the (ordered) Fourier transform coefficients of the arterial function • This was found empirically by us, but turns out to be well-known effect from radar studies in 1991

27. Engineering “convolution” theory indicates “we are right” • Consider convolving (or deconvolving) two signals • LINEARITY PROPERTY: • Double the amplitude of one input – doubles output amplitude – no change in shape • POSITION INDEPENDENT: • Shift position of input by amount x. Output will shift position by amount x – no change in shape • Theory indicates that a “proper” deconvolution algorithm should be “delay independent”

28. SVD well known – Why is it not working in DSC MR studies? • Actually neither SVD nor FT have ever really worked in one sense – but nobody says it. • Deconvolution works by deconvolving the “effect” by its “cause” – and a “cause” signal always arrive before the “effect. • The “tissue” is not the “effect” that is produced by the “arterial” signal, but is the effect of the “injection into the arm. • Thus it is physiologically possible for the tissue “effect” signal to arrive BEFORE the “proxy” arterial “cause” signal.

29. SVD and FT deconvolution have different properties NEGATIVE POSITIVE TIME TIME • The FT deconvolution algorithm has “cyclic” properties • In the presence of a delay, any “negative time residue function signals” are wrapped around (aliased) to become a false “high time signal”. • However, PROVIDED THERE ARE NO TRUE HIGH TIME SIGNALS, we can unwrap and get “correct answer” . “UNWRAPPED” HIGH TIME SIGNAL

30. SVD and FT deconvolution have different properties NO NEGATIVESIGNAL ALLOWED • The SVD deconvolution algorithm was not being implemented with “cyclic” properties • No negative time signals are allowed. • But that “energy” must go somewhere – and it goes into boosting the early residue function peak • For a zero delay -- This boost counterbalances the signal loss from noise filtering • SVD acts as “the better algorithm” when incorrectly implemented • However, the “improvement” is very unstable “MISPLACED” NEGATIVE ENERGY

31. Big fight with reviewers • First of all reviewers would not accept that • There was an effect or • that our theory was valid • Later, when somebody “well known” published a circular SVD implementation, we were told by the reviewers that “since a better algorithm had already been published, then ours should not be published”. • Fortunately the editor stepped in and we published our improved SVD algorithm (as a short note), but we never recovered the precedence. • New papers are still showing misunderstanding of the significance of what we have explained about delay issues.

32. 0ther implications • All that “dispersion effect” is also an artifact Using a “delay” insensitivedeconvolution approach shows dispersion effect is much smaller than described earlier

33. Biggest issue remaining • We are continually changing our algorithms as we better understand the “engineering” theory. • How can we (easily) check that the changes we are making are not having an unexpected effect in “previously working” parts of our code. • In the business world, a new concept in software development is “Agile” – a light weight, low-document producing development process. • A key element of “Agile” is test driven development and an automated testing framework – two issues useful in different ways

34. Comparing Test-Driven-Development with “the Scientific method” (Mugridge 2003) The scientific method Test-Driven Development (TDD) We don’t need to change our thought processes very much to switch to TDD. Biggest issue is having to change our work habits and beliefs. As a physicist I had been trained to “think about tests and testing issues” before coding, therefore formalizing those thoughts into real tests is not too hard (30% of the time)

35. Main difference between TDD and “normal” software development TDD approach -- Many initial testsused to describe “ideas” – later used for “regression testing” when ideas change Standard water-fallmethod. Tests often forgotten in time crunch.

36. We have been successfull in applying TDD to biomedical embedded systemsF. HuangA. TranA. Kwan

37. Current research (J. Qiao) • How do you move the “idea behind applying the scientific method” in planning your research procedure over into“using test-driven development” in planning the software code (Matlab) you need for that research procedure and later use those tests when commercializing onto the biomedical instrument?

38. Conclusion • When starting your research project – make sure you understand your goals. • Be prepared to change your goals as opportunities arise. • Try to duplicate the results in existing literature, but remember, you are “engineers” and have a different knowledge set that many of the “clinical” people • Be prepared for unexpected results. • Have an automated testing approach so that you can duplicate your (software) results easily and provide easily repeatable evidence that “everybody else has “not handled things correctly.