I recently just started tinkering with this exact problem myself. I don't know how I'm going to get it completely universal, but I'm trying an implementation that should be able to handle up to 6th degree polynomials. The polynomial synthetic division routine is rather simple algorithm-wise. Once I get to that bridge I can supply further updates, but right now I'm trying to deal with factoring P and Q. I'll probably go about that by decrementing a variable, say variable H, by 1 each time and using my Ti-84's remainder( function to see if it's a factor compared to variables A and G. Any better ideas on how to factor a number? I don't suppose I need to implement Descartes' Rule of Signs because this is a computer (albiet only 16 kHz), but to save time, I'm struggling to figure out an implementation. I might do something where I test the term before and the term after to see if the value of the coefficient is greater than zero and see if there's a change. It should be relatively easy to do the "negative x" routine to find the possible number of negative roots. But that increased speed from not having to try all the possible roots might be sacrificed to program size if I make the whole Descartes Rule routine.
Programming calculators is fun, but laborious because there's no copy-paste and typing is a pain.
P.S. I also have a quadratic factoring program that uses the completing the square method and can give values for the forms (x-a)^2, x^2 + bx + c = d, and (x-a)(x-b) to completely eliminate any work from the problem (unless it turns out to be radicalized answer, and I haven't implemented a radical simplification routine into it so it gives me awkward decimals). Basically, I can fake showing my work. I won't do that IRL of course, but it's nice to have the calculator verifying my math as I go.