I think you will be able to map the contents of the square into the quad provided the transformation that the projector has unintentionally applied is actually projective. This is how I would go about things in that case:

I presume you can calculate both the maximum sized square that the projector can display without clipping the image, and where this square currently appears. Although these images are in different planes we can ignore this and pretend that both planes are the same.
ABCD is the original square A'B'C'D' is what actually appears - I've shown it as an isosceles trapezium but in the worst case scenario it could be any convex quadrilateral.
Draw the largest possible square (or just a large enough square you can calculate conveniently) inside A'B'C'D' and call it A''B''C''D''

Now decree that ABCD and A''B''C''D'' have the same coordinates. You can certainly do this because they are similar squares. Now calculate the coordinates of A'B'C'D' relative to this coordinate system. Now you decide that the projector ought to have displayed the square ABCD your image in A''B''C''D'' whereas it actually displays it A'B'C'D', so that it has apparently applied the unique projective transform that turns A''B''C''D'' into A'B'C'D'.

You can calculate this transformation using the method I suggested in my last post.
Suppose the coordinates of A''B''C''D'' are

A''=(0,0,1)
B''=(1,0,1)
C''=(1,1,1)
D''=(0,1,1)

(Whatever they are they should be the same as your coordinates for ABCD - If you were using ordinary 2D coordinates you just assign 1 as the third coordinate).

Maybe the coordinates for A'B'C'D' are something like

(-0.2,-0.3,1)
(1.3,-0.2,1)
(1.5,1.3,1)
(-0.5,1.4,1)

(where now I am pretending that the distortion also spoiled horizontal lines)
The Z coordinate should certainly be the same as it was for the square).

Now calculate the matrix for the following transformations
(1,0,0)->A''
(0,1,0)->B''
(0,0,1)->C''
(1,1,1)->D''

Call this M

Next calculate the matrix for

(1,0,0)->A'
(0,1,0)->B'
(0,0,1)->C'
(1,1,1)->D'

Call this N

The transformation the projector has notionally applied to our undistorted square is

N*M^-1 (i.e. first M^-1 then N)

So to undistort it we need to apply M*N^-1.

To apply this to a point (x,y) (which we assume will be (0,0) for A, (1,0) for B and so on)

Turn (x,y) into (x,y,1) Compute M*N^-1*(x,y,1) (treating this as a column vector).
Now you will have a coordinate X',Y',Z' where Z' is most likely close to 1 but not exactly 1. Now divide X',Y' by Z'. That is the adjusted coordinate you should use for computing your image.