Before you say it as simple as uninstalling any installed program from Programs and Features or Add/Remove Programs in Control Panel, let me tell you you’re right but you need to disconnect your Google account to Google Drive Windows app before that which stops Google Drive folder from syncing with Google Drive on web after that, you can uninstall Google Drive from Control Panel.
Uninstalling Google Drive Windows App from Computer
1. Right click on Google Drive icon in system tray and select Preferences
2. Now on Preferences Window click on “Disconnect account”, click Yes to the dialog displayed.
3. Now right-click on Google Drive tray icon and select “Quit Google Drive”.
4. Now go to Control Panel>Programs>Programs and Features select Google Drive and click uninstall button.
Once uninstall is finished, you can able to delete Google Drive folder and its contents located in the directory “C:\users\username”.