TeamViewer adds an extra button to the titlebar of all active applications. This button with a double-headed arrow is called the QuickConnect button in TeamViewer jargon. It is quite irritating to see this button (and to click it by mistake) if you do not really want to be bothered about the QuickConnect feature.
The option to hide this button is hidden away in TeamViewer settings. Navigate through Options → Advanced → Show advanced options → QuickConnect button → Configure and uncheck Show QuickConnect button.
Tried with: TeamViewer 7.0.12313