In this thesis, I investigate the aspect of user interaction in Augmented reality for task support in a handheld device. I focus on a handheld device as it becomes a promising platform for many AR applications due to accessibility and popularity nowadays. I divided the user interaction into two groups based on role in task supporting: 1) Input the virtual instruction by an expert user, and 2) Receive and follow the virtual instruction by a novice user. In Chapter 2, I address the problem of an expert user for inputting the virtual instruction. In the task supporting, in-situ object placement and manipulation required to perform. Thus, an intuitive object manipulation technique is needed for this task. I present SlidAR+, a Gravity-Aware 3D Object Manipulation for Handheld Augmented Reality. SlidAR+ is a method for controlling the position and orientation of virtual objects in HAR. Next, I present the results of experiments by comparing SlidAR+ and a state-of-the-art method to evaluate the performance of SlidAR+. In Chapter 3, I investigate the effect of the latency of the handheld device's camera on task performance. As for the novice user, following the instruction correctly is fundamental. However, looking through a camera lens in the video-see through displays affects users' performance due to distortions in visual representation and hardware performance. For a portable device such as a handheld with limited hardware, performance will inevitably become slower when more content and information are added. To address this, I investigate the effect of the latency of the mobile phone's camera on task performance. I conducted two small studies: 1) To see which levels of latency users start to notice, and 2) How does latency affect the task performance. To summarize in this thesis, I present two works: 1) SlidAR+: Gravity-Aware 3D Object Manipulation for Handheld Augmented Reality and 2) an investigation of the effect of latency on 2D display for micro-task.