Backend work, or FrontEnd work?

I think most programmers prefer the backend, some frontend work is fun (writing shaders for example) but things like GUI design is really an artists/designers job, not a programmers and that is why we have scripting languages and wysiwyg GUI design tools etc.

The definitions of frontend/backend are a bit grey.
I'd personally assign code to either category as to whether a user (whether that's the player, or another dev department) can interact with the code directly.

e.g. When I'm writing shaders, I'd classify it as backend work, because it's low-level specialized programming internal to the game, and no one can interact with it directly. The customers of the work are the artists, and they interact with them via a very long chain:
* Maya Material Tools -- front end
* Maya Exporter Plugin -- back end
* Engine Data Compiler Plugin -- back end
* Engine Data Compiler GUI -- front end
* Game Data Loaders -- back end
* Game Render System -- back end
* Game Shaders -- back end

Regarding front-end programmers vs back-end programmers, different people are better at different roles. When designing the GUIs, HUDs and menus on a professional game, that should all be designed by a graphic designer or "UX designer", not a programmer. However, this designer would then work hand-in-hand with a UI-specialist programmer to implement their designs, who in turn would work directly with a back-end rendering programmer to get the rendering APIs that they need, etc...

That said, I enjoy isolated low-level systems tasks (where the client of my work is another programmer) more than tasks that require me to make something usable directly by non-programmer clients ;)

personally i hate the front end, positioning buttons, making sure stuff appears correctly, that things will disappear when another menu is opened, making sure everything is workable is annoying to me, it just feels like a tedious task to make sure it's all working right.

It depends what I'm doing in the front end. Starting screens/UI from scratch I actually enjoy. You get to see results right away, and you get to [usually] make a rather robust small system quickly. Doing maintenance on front end code sucks though. Generally you're working on a piece of UI that's more than likely had everyone ever involved with the game changing some piece of code, and it's probably been redesigned multiple times (if you are lucky through multiple paradigm shifts in the front end's functionality). In the latter case you're usually stepping through 10s to 100s of lines of code just to figure out why a border isn't showing up or, like you said, moving a button 5 pixels to the right.

I work at a place where these are actually two different jobs, and they really like what they do (and couldn't be arsed to do the other's job) so I'm guessing you can enjoy both.
That said, I prefer fullstack programmers, so I wished you could like both ;)