First of all I would ask myself the question, would I be happy with the two stumps in my garden providing they are cut low and neatly to the ground. If the answer is yes, you could use them to place plant pots etc on.
If the answer is no, I would place a job on this site and ask local reputable businesses to provide quotes to find the cheapest option for them to come and remove the stumps using a stump grinder.
If your happy to deal with the re-growth problem yourself I would research eco-plugging and not resort to using diesel etc to poison the stump as this has adverse environmental issues. I would ask for someone to quote to come and remove the re-growth and eco-plug the stump. By eco-plugging the stump around the edge this will prevent any future re-growth. Thus solving your re-growth issue but the stump will remain.

I removed a yucca recently, including the roots, they are shallow rooted making complete removal easy. If you got this far in the removal process, then you're probably more than capable of completing the process. Firstly, don't cut the stump any lower, the leverage will be helpful, secondly, get a digging tool, ideally a mattock and cut the root about six inches from the stem. Get right under the stem, then test for any larger roots holding it in place. Continue to dig/chop roots and eventually it will come out.

3 options from cheapest to most costly.
remove the stumps by hand digging them out, hard work but completely free.
Use Eco Plugs drilled into the stump poisoning the stump and waiting for it to decay to easily be removed, a timely process but can be as cheap as £10.
Or have the stump removed with a grinder which will cost more but requires no effort from yourself.