Re: STG2

[Jon Elson <jmelson-at-artsci.wustl.edu>]

JohnDRoc-at-aol.com wrote:

> I wiped the HD yesterday, reinstalled RH6.0, installed RTL2.0 then installed
> the May release of EMC - still using the stg2mod.o motion controller.  There
> was no difference.  It's actually kind of weird, when I measure DAC0 it will
> creep up slowly (ramping up), but DAC1 and DAC2 will both spike.  In fact, no
> matter which direction button I press, DAC1 would still go positive!?

This is with EMC?  Yes, I guess so.  Well, I don't believe anyone has
run a servo machine with anything past March, 2000.  I am still running
the 20-Dec-1999 version because of SERIOUS problems with all
later versions!  It appears that nobody at NIST has a servo machine right
now, and the STG2 driver was written at NIST without the card, and a
guy at a different location gave them feedback.  So, it is no surprise that
you are having trouble.

The ramping up might be due to very slow creep of the axis, causing it to
develop a few encoder counts away from the commanded position.

Spikes - yes, I know.  there is either a design error in the STG cards,
the chips they use for encoder counters, or the driver software.
I still don't know where this problem is coming from, but I DO know
what is going on, to some extent.  First, the position the machine is
at when EMC starts is apparently preset to a digital count of zero.
If the machine moves back (in the minus direction) one encoder count,
the number goes negative, to a binary all-ones value, which equals
a 24-bit signed integer of -1.  This requires all flip-flop bits in the
encoder counter to flip simultaneously.  It appears that this either
causes excessive noise in the chip, or that the propagation of the
23 borrows takes so long that the latch that holds stable data for
the computer to pick up does not latch the correct count value.
The actual running count is OK, but one sample of the latch is
corrupt.  It gets a position reading that is WAY far off, causing
large (like full-scale) spikes in the DAC output, as the positioning
loop attempts to correct the error.  This only happens when the
machine is left at, or returned to the position it was at when EMC
was started.  It is called the 'clicking' problem, and was solved at
one time by comparing the fresh sample to the last sample, and
if the difference is too great, the fresh sample is discarded.
This fix appears to have been lost or broken between 20-Dec-1999
and the Mar-2000 release.

I had to substantially detune my servo amps so they wouldn't get
an overcurrent fault from these transients.  I did manage to capture
one with the built in data logging, which proved where the problem
was coming from - namely the encoder counters.

> (Another weird thing is that if you change the settings on one axis, it seems
> to change it for all axes.)  I would suspect the board, but there are
> separate opamps for 0&1, 2&3, etc.  If 0 works, why not 1?  Plus I've been
> able to manually set voltages on it.  I think something may be getting put
> together (the way the base address is appended to the DAC and the counts)
> wrong when it writes to the DAC.

Yes, if this odd behavior is NOT seen with the STG diagnostics, then this
is most likely a software problem, not the STG card.

>  There is supposed to be a way to run in
> non-realtime and log the output to the DAC's, but I have no idea how to do
> it.

You don't have to go out of real-time.  The standard EMC setup has a
diagnostics pull-down menu, with a logging entry.  You can select
how many samples to skip between recorded samples, and how many
samples to record in a ring buffer.  The axis that is highlighted on the
main screen is the one for which data is recorded.  You should be able
to start logging, and then stop it just after an event of interest ocurrs.
You will get a file that is a tabular list of time, and several parameters,
such as position and DAC output.  There is a utility called xgraph
(available on the NIST FTP site) that can process these files into
instant graphs on the screen.  In some EMC versions, the perl scripts
that remassage this data into xgraph's favorite form don't work, I can
help you with that if yours doesn't work.  You could also just scan
the file for what you are looking for.

>  Furthermore, I've spent about 4 weeks trying to debug this thing, which
> is equal to 4 weeks that I haven't done any "real" work (i.e. I'm gonna
> starve soon, if I don't).  I think I'm going to see if the fellows up at NIST
> would be receptive to trying the board themselves.  I may even get a wild
> hair and drive it up myself, so I can see it work.

Going up to NIST for a visit could be a VERY educational experience.
Meeting Fred and Will should be quite an experience.  I've talked to
Fred on the Phone, he has been ENORMOUSLY helpful when I was
having trouble with these problems.  Will is more involved with the
deepest parts of the motion and interpreter sections, but has also
been helpful through email.  I know that Matt Shaver has gotten a LOT
of help from them, and does some testing and feedback for them
in return.  One possibility is that the card could be tried at Matt's,
where he might have a servo machine operational shortly.  I don't
know what NIST would do with the card, but they may have some
servo bench-test gear to try it on.  If you go, take your entire
computer, if possible.  It could be some timing glitch that causes
problems only on your computer (although I doubt it).

But, my first suggestion is to get the earliest software version that is
supposed to support the STG2 card.  That is your best chance.

Good luck,

Jon