Hey Gungeoneers,
We're releasing a new beta today on the "beta_public" branch that includes a Save and Exit option at the end of each floor. Please try it out and send any feedback or bugs you find our way (via the Steam Discussions)! For now this beta is only available on Windows and in English, but will be released fully with the Supply Drop update, coming soon!
If you're interested in joining the public_beta, you can opt in from Steam by going to Enter the Gungeon > Properties > Betas and selecting "beta_public" from the drop down). To verify you have the beta, look for "1.0.11 save beta" at the bottom right of the title screen. (If you don't see beta_public option, restart Steam to update the list)
At the end of each floor in the final elevator room, there will be a big red button next to the elevator; hit it and the elevator will be replaced with a cyrochamber, which when entered, will save your current run and put you back on the title screen. You can resume your run from the title screen at a later time and you'll start at the beginning of the next floor.

No, whenever you die or exit the game in any other way, your save is deleted. This is only a system to allow you to take breaks during long runs, not a way to save and retry until you succeed.
It's coming! We're in the balancing/polishing/bugtesting phase now, and this save beta is one more step toward its release!