Graphics with Canvas, SurfaceView , and multitouch processing (panning and multitouch zoom)

1. Graphics with Canvas, SurfaceView, and multitouch processing (panning and multitouch zoom) www.eecis.udel.edu/~bohacek GraphicsWithCanvas_2012.pptx

2. Approaches for Graphics • Load image from /res/drawable • Best for static images • OpenGL ES • 3D graphics (i.e., transforms such as spin can be applied to graphical objects) • Best for game-type animation • Draw on Canvas or SurfaceView • Canvas for drawing within the UI thread • SurfaceView is faster, and better for detailed graphics • Note, if your main thread take too long, the OS will kill it, and it will be difficult to debug.

3. Drawable shapes • Make new app, edu.udel.eleg454.Graphics1 • In onCreate is setContentView(R.layout.main) • Instead of the view generated by R.layout.main, we use our own, which extends View • In Graphics1 class, add • private class MyView extends View { public MyView(Context context) { super(context); } @Override protected void onDraw(Canvas canvas) { ShapeDrawablemDrawable = new ShapeDrawable(new OvalShape()); mDrawable.getPaint().setColor(0xff74AC23); mDrawable.setBounds(10, 10, 310, 60); mDrawable.draw(canvas); } } • Then, in onCreate, replace setContentView(R.layout.main); with setContentView(new MyView(this)); • Run • Besides OvalShape, ArcShape, PathShape, RoundRectShape, Shape, and BitMaps

4. View Widget • The previous method required us to replace setContentView(R.layout.main); • This resulted in the entire view being controlled by our view object • E.g., we could not have a button in the view where we place the button with the layout editor • To fix this, we add a view widget • Move MyView to separate class • Make new class • In package explorer, under /src • Find edu.udel.eleg454.TestGraphics1 • Right click on edu.udel.eleg454.TestGraphics1 • Select: new->class • Dialog opens • Name: MyView • Superclass: View (then select browser to get full name: android.View) • OK • Move functions from private class MyView to this new MyView • Move public MyView( • MoveonDraw • Also, in MyView add public MyView(Context context, AttributeSetattrs) { super(context, attrs); }

5. View Widget • Go to main.xml graphical layout editor • Drag button to the screen • Leave id as button1 • Go to main.xml (not the editor) • Find second <Button …> </Button> • Before <Button>, add • <edu.udel.eleg454.Graphics1.MyView android:layout_width="wrap_content" android:layout_height="wrap_content“ android:id="@+id/View01"></edu.udel.eleg454.Graphics1.MyView> • Note that edu.udel.eleg454.TestGraphics1.MyView is the name of the separate class. If another name is used, then this name should be changed • Save and go back to graphical view. There should be a box labeled MyView. Drag the box to make it larger • Run

6. Canvas Drawing • Canvas has many drawing functions, e.g., drawPath(Path path, Paint paint) • In onDraw, add the following • Path: sequence of graphical objects • Path path = new Path(); • Make line between two points • path.moveTo(10,10); // starting place • path.lineTo(160,160); • add circle somewhere • path.addCircle(160,160,20, Path.Direction.CCW); • Paint – for setting color and line width • Paint paint = new Paint(); • paint.setDither(true); • paint.setColor(Color.RED); • paint.setStyle(Paint.Style.FILL_AND_STROKE); • paint.setStrokeJoin(Paint.Join.ROUND); • paint.setStrokeCap(Paint.Cap.ROUND); • paint.setStrokeWidth(10); • Draw view canvas.drawPath(path, paint); • run

7. Change graphics • At the end of TestGraphics1Activity.onCreate • MyViewmyView = (MyView) findViewById(R.id.View01); • Button button = (Button)findViewById(R.id.button1); • button.setOnClickListener(new View.OnClickListener() {}); • Let eclipse add unimplemented methods • In onClick, add • myView.redraw(); • In MyView • Add class variable • int radius = 20; • In onDraw, change • path.addCircle(160,160, 20, Path.Direction.CCW); • To • path.addCircle(160,160,radius, Path.Direction.CCW); • Add function • public void redraw() { • radius = 80; • invalidate(); // this is needed to force redraw • } • run

8. notes • Use invalidate() to force redraw • In Graphics1, add variable • MyViewmyView; • In Graphics1.onCreate(), add • myView = (MyView) findViewById(R.id.View01); • Then myView.invalidate(); will force redraw • Is canvas documentation for more graphics • E.g., drawBitmap has several functions • Avoid declaring and setting variables in onDraw, instead, setting them elsewhere and access them from draw • Use invalidate() to force redraw • Use SurfaceView for faster screen drawing

9. SurfaceView • Faster • You can draw on a SurfaceView from other threads, not just to UI thread • When drawing with the UI thread, if the drawing takes a long time, then everything else must wait for the drawing to complete, • e.g., the user cannot press any buttons • If you put a long activity in the UI thread, a message will pop up saying that the app has stopped responding • If you put a long drawing activity when starting the app, the system just kill it (thinking that it did not start correctly) • But, SurfaceViews are not transparent, nothing behind the view can be seen • Differences • In Canvas approach, your onDraw function is called and has argument canvas. You can draw on this canvas. You can force a redraw can calling invalidate. • Invalidate will result in onDraw being called from the UI thread • With surfaceview, you get a canvas and can draw on it whenever you want. Usually you draw on it from a new thread • E.g., You start the thread from the UI thread

10. SurfaceViewFun • Make a new app, SurfaceViewFun and package name edu.udel.eleg454.SurfaveViewFun • Make new class • Right click on edu.udel.eleg454.SurafceViewFun • Select New -> class • Name: MySurfaceView • SuperClass: SurfaceView • Go to res/layout/main.xml • Open graphical view • Click on “Advanced” • Drag SurfaceView • In xml view, find the <SurfaceView …. • Change <SurfaceView … to <edu.udel.eleg454.SurfaceViewFun.MyView • Go back to graphical view and check that the surfaceView is now labeled MySurfaceView. If not, then something is wrong.

11. MySurfaceView • Open MySurfaceView • Eclipse will ask to add some unimplemented functions. Add all three of these • public MySurfaceView(Context context, AttributeSetattrs, intdefStyle) • public MySurfaceView(Context context, AttributeSetattrs) { • public MySurfaceView(Context context) { • Each on has content • super(context, attrs); // eclipse migth add this part • ini(context); // a function to make

12. Change • public class MySurfaceView extends SurfaceView • To • public class MySurfaceView extends SurfaceView implements SurfaceHolder.Callback • Add unimplemented functions • Add member variables • SurfaceHoldersurfaceHolder; // needed fro drawing • MyThreadmyThread; // will make MyThread next

13. Thread class • The whole reason to use a surfaceView is to draw on a different thread then the UI thread. Let’s make a thread class. • Make new subclass in MySurfaceView and extend Thread, i..e, add • class MyThread extends Thread {}; • This will be the thread we use for drawing • We will draw an oval, by the drawing is slightly animated • In MyThread, add member variables • SurfaceHoldersurfaceHolder = null; // this a key variable as it allow use to get a canvas • boolean done; • long startTime; • Canvas canvas; • RectFrectForOval = null; • float duration = 5*1000; • Paint drawingPaint; • float arcSweep;

14. MyThread member functions • Constructor • public MyThread(SurfaceHolder _surfaceHolder) { • surfaceHolder = _surfaceHolder; • drawingPaint = new Paint(); • drawingPaint.setColor(Color.BLUE); • } • We will draw an oval, but it will be slightly animated • This is the function that will run in the thread. • @Override • public void run() { • Log.e("surface","running thread"); • // some initialization • setOval(); • startTime = System.currentTimeMillis(); • arcSweep = 0; • done=false; • while (!done) { // draw until done • updateArcSweep(); • drawCurrentArc(); • } • Log.e("SurfaceViewFun","MyThread.run has finished"); • }

15. Add the following functions to MyThread • public void setOval() { • rectForOval = new RectF(10,10,300,600); • } • void updateArcSweep() { • long currentTime = System.currentTimeMillis(); • arcSweep = (float) ((currentTime-startTime)/duration*360.0); • if (currentTime-startTime>duration) { • done = true; • arcSweep = 360; • } • } • void drawCurrentArc() { • canvas = surfaceHolder.lockCanvas(null); // must lock before drawing • canvas.drawColor(Color.BLACK); // clear the screen • canvas.drawArc(rectForOval, 0, arcSweep, true, drawingPaint); // draw new stuff • surfaceHolder.unlockCanvasAndPost(canvas); // must unlock when done • }

16. MySurfaceView • Initialize (this function is called by each of the MySurfaceView constructors • public void ini(Context context) { • Log.e("SurfaceViewFun","ini"); • surfaceHolder = getHolder(); // MySurfaceView extends SurfaceView, which has member function getHolder. We use the surfaceHolder to get the canvas that we will draw on • surfaceHolder.addCallback(this); // MySurfaceView implements SurfaceHolder.CallBack. This allow MySurfaceView to get message about when the surface is ready for drawing and whether the surface has change (e.g., change orientation) • myThread = new MyThread(surfaceHolder); // make thread • setFocusable(true); • } • In surfaceChanged, add • Log.e("SurfaceViewFun","surface changed. Width: "+width+" height: "+height); • Start thread • Since we extend SurfaceHolder.Callback , we implement surfaceCreated. • When this function is called, the surface is ready for drawing (the surface might not be ready for drawing when the SurfaceView constructor is called. We need to wait for the surfaceCreated function) • In surfaceCreated, add • Log.e("SurfaceViewFun","created"); • myThread.start(); // starts thread. Will lead to MyThread.run being called in its own thread • Run it

17. playing • Compare directly writing onto canvas vs using a surfaceview and a thread • In surfaceCreated,change • myThread.start(); • To • myThread.run(); • Change • class MyThread extends Thread { • To • class MyThread { • Change • @Override • public void run() { … • To • //@Override • public void run() { …. • ]that is, comment out the override statement • So now there is no thread, we writing from the UI thread. • Run it • Undo what we did so it draws in a thread

18. Multi-touch and zoom • View objects process touches. • We need to implement this processing to capture multi-touch. • Also, to process some touches, we need GestureDetectors to help • In summary • Override onTouchEvent • Extend ScaleGestureDetector.SimpleOnScaleGestureListener • Extend GestureDetector.SimpleOnGestureListener • There is a bunch of code, but only a couple of critical spots • In MySurfaceView::ini, add • iniTouchHandling(context); • At the end of MySurfaceView, add • private float mPosX =0, mPosY = 0; // will indicate how much we have panned. Use these to adjust the graphics • float mScaleFactor = 1.f; // indicate the scalling. Use this to adjust graphics • private float mLastTouchX; • private float mLastTouchY; • private static final intINVALID_POINTER_ID = -1; • private intmActivePointerId = INVALID_POINTER_ID; • GestureDetectormTapListener; • ScaleGestureDetectormScaleDetector; • public void iniTouchHandling(Context context) { • mScaleDetector = new ScaleGestureDetector(context, new ScaleListener()); • mTapListener = new GestureDetector (context, new TapListener()); • }

19. onTouchEvent • @Override • public booleanonTouchEvent(MotionEventev) { • // let our gesture detectors process the events • mScaleDetector.onTouchEvent(ev); • mTapListener.onTouchEvent(ev); • final int action = ev.getAction(); • switch (action) { • case MotionEvent.ACTION_DOWN: { • final float x = ev.getX(); • final float y = ev.getY(); • // Remember where we started • mLastTouchX = x; • mLastTouchY = y; • mActivePointerId = ev.getPointerId(0); • break; • } • //More on next slide

20. onTouchEvent (continued) • case MotionEvent.ACTION_MOVE: { • final intpointerIndex = ev.findPointerIndex(mActivePointerId); • final float x = ev.getX(pointerIndex); • final float y = ev.getY(pointerIndex); • if (!mScaleDetector.isInProgress()) { • // Calculate the distance moved • final float dx = x - mLastTouchX; • final float dy = y - mLastTouchY; • // Move the object • mPosX += dx; • mPosY += dy; • // Remember this touch position for the next move event • mLastTouchX = x; • mLastTouchY = y; • // Invalidate to request a redraw • // invalidate(); // use this is a regular canvas is being used • myThread.setOval(); • if (myThread.done==true) • myThread.drawCurrentArc(); • } • break; • } The current amount that we have panned • With new values of mPosX and mPosY, we need to update the graphics • We can remake the oval • If we are still drawing, the the new oval will be drawn • If we have finished drawing, tehn we need to redraw • If we are not using a surface view (i.e., we are directly drawing on the canvas, then in order to have the new values of mPosX and mPosY be shown, we need to call invaliate for the view

21. onTouchEvent (end) • case MotionEvent.ACTION_UP: { • mActivePointerId = INVALID_POINTER_ID; • break; • } • case MotionEvent.ACTION_CANCEL: { • mActivePointerId = INVALID_POINTER_ID; • break; • } • case MotionEvent.ACTION_POINTER_UP: { • // Extract the index of the pointer that left the touch sensor • final intpointerIndex = (action & MotionEvent.ACTION_POINTER_INDEX_MASK) • >> MotionEvent.ACTION_POINTER_INDEX_SHIFT; • final intpointerId = ev.getPointerId(pointerIndex); • if (pointerId == mActivePointerId) { • // This was our active pointer going up. Choose a new • // active pointer and adjust accordingly. • final intnewPointerIndex = pointerIndex == 0 ? 1 : 0; • mLastTouchX = ev.getX(newPointerIndex); • mLastTouchY = ev.getY(newPointerIndex); • mActivePointerId = ev.getPointerId(newPointerIndex); • } • break; • } • } // and switch statement • return true; • } // ends onTouch

22. ScaleGestureDetector.SimpleOnScaleGestureListener • private class ScaleListener extends ScaleGestureDetector.SimpleOnScaleGestureListener { • @Override • public booleanonScale(ScaleGestureDetector detector) { • mScaleFactor *= detector.getScaleFactor(); • // Don't let the object get too small or too large. • mScaleFactor = Math.max(0.1f, Math.min(mScaleFactor, 5.0f)); • //invalidate(); // use this for a regular canvas (i.e., not a SurfaceView) • myThread.setOval(); • if (myThread.done==true) • myThread.drawCurrentArc(); • return true; • } • } • Since mScaleFactor has changed, the graphics should be updated • Also, if we are drawing directly on the canvas, then we need to invalidate sot natonDraw is called

23. GestureDetector.SimpleOnGestureListener • private class TapListener extends GestureDetector.SimpleOnGestureListener { • @Override • public booleanonDoubleTap(MotionEvent e) { • Log.e("SurfaceViewFun","double tap "+e.getX()+" "+e.getY()); • return true; • } • @Override • public void onLongPress(MotionEvent e) { • Log.e("SurfaceViewFun","got long press at location x="+e.getX()+" y="+e.getY()); • } • @Override • public booleanonFling(MotionEvent e1, MotionEvent e2, float velocityX, float velocityY) { • Log.e("SurfaceViewFun","fling: started at ("+e1.getX()+" ,"+e1.getY()+"). Ended at ("+e2.getX()+" ,"+e2.getY()+"). With velocity ("+velocityX+" ,"+velocityY+")"); • return true; • } • @Override • public booleanonScroll (MotionEvent e1, MotionEvent e2, float distanceX, float distanceY) { • Log.e("SurfaceViewFun","scroll: started at ("+e1.getX()+" ,"+e1.getY()+"). Ended at ("+e2.getX()+" ,"+e2.getY()+"). With total distance ("+distanceX+" ,"+distanceY+")"); • return true; • } • } Our app does not use these touches. But if you want them, here they are

24. Make graphics use mPosX, mPosY, and mScaleFactor We should make sure that no other thread is reading RectF ad we are resetting it. We synchronized with surfaceHolder • Need to change the oval • public void setOval() { • synchronized(surfaceHolder) { // make sure that we are not drawing while updating the oval • rectForOval = new RectF(10*mScaleFactor+mPosX,10*mScaleFactor+mPosY,300*mScaleFactor+mPosX,600*mScaleFactor+mPosY); • } • } • drawCurrentArc also need to be synchronized • void drawCurrentArc() { • canvas = surfaceHolder.lockCanvas(null); • canvas.drawColor(Color.BLACK); • synchronized(surfaceHolder) { • canvas.drawArc(rectForOval, 0, arcSweep, true, drawingPaint); • } • surfaceHolder.unlockCanvasAndPost(canvas); • } • Run mPosX, and mPosY translate the oval mScaleFactor scales the oval