Selfmade (modified) variable voltage AC using step down converter

Let's suppose that you need to plug something in that requires less voltage than any of your spare ACs give out. I had to make a thingy work that needed 3.5-4V max, but I only had 5,9 and 12 volts ACs at my disposal. What do you do in that case? Well of course you could order voltage-variable AC on ebay, that comes with a lot of plugs too! But then you'd have to wait for shipping and pay somewhat 10$, but you already have a lot of ACs, why not upgrade them? Besides, the majority of variable ACs are not really adjustable, they have 3-5-9-12 switch, and that's it. So you can't really use thouse if you need exactly 4.2V output.

The things you'll need:

Replacing Sony reader PRS-500 battery

I inherited an old Sony e-ink reader Sony PRS-500. 

I'm unsure of how old this thing is, but I would argue about 6 or 7 years. Most of that time it was lying around unused and forgotten, but was actively used in the first couple of years after purchase.

Stock battery: LIS1356HNPAQ (SY6), 850mAh, 3.7V;

After 7 the battery deteriorated, and I could only go through 100-150 pages before it shut down. 

Moreover, it was missing the original charger, but that issue was resolved swiftly and easily. I simply had to find appropriate center-positive plug, and solder it with red and black wire of a USB-plug. The original charger was supposed to be 5.2V, USB's voltage is 5.0V, which is not much of difference.

Now, let's move on to the battery replacement. There are two options I could find: