I think that this property  determines if published documents are
indexed. I am not 100% sure though. Does this prop get set  if you use
the gui to mark a folder as not indexed?

This property  is only exposed in the Schemas folder, not on the document
itself. If I open the schema properties on the document in the WSS
Explorer, I can see the property but not modify its' value. If I open
the Schemas folder, then I see the property and can modify it its value
from 1 to 0. Only 1 or 0 are accepted, AFAICT.

Where do I use a GUI to mark a folder as not indexed?

Under content sources, can you mark a folder to be excluded from the
indexing? Anyway, I did a quick look at the wss. I was incorrect with my
previous answer. It looks like that prop is only on property  definitions
that is under the schema folder. Does that appear correct to you? IMy
thought is that this determines if the property is included in the
index. You might try changing one from a 1 to a 0 and do a full index
and see if your searches can not find any results based on this field.
This is just based on a quick look a the wss. I am working on getting
out to the airport this afternoon and I could only spare a quick look.
Also note, if you do not already know, that some searches use the full
text indexes while some use the WSS searching. So it would probably be
possible that even if this prop does what I think it does, a search
using the WSS engine would still find it. I think to avoid the WSS
search, use a deep search or include some actual text along with the
property search.
If you do this test. I would be curious to know the results.

What I'm trying to do is see if I can get checked in, but not published
documents indexed  by changing the value of this property.

No, I don't think that this will do the trick. You might try going to
content Sources and adding a path to the shadow folder. I think the
reason they are not indexed, is because the shadow folder is not
indexed.