Review of the EMC conference at NAMES 2003 (long)

["Steve Stallings" <stevesng-at-newsguy.com>]

    Review of EMC Conference at NAMES 2003

Many developers and active users of EMC gathered at
the recent NAMES (North American Model Engineering
Society) show in Detroit, Michigan, USA over the
weekend of April 26 and 27, 2003. There was also
a conference on the future direction of EMC held
on the Monday after the show.

At the show there were half a dozen systems up
and running EMC with the owners explaining EMC
to the show visitors. The display area was donated
by the NAMES show sponsors and organized by Roland
Friestad of Cardinal Engineering. The purpose of
the displays was to inform model builders and other
metalworking hobbyists about CNC machine tools and
how to build them for their own use. In previous
years this display and associated seminars helped
inspire Bill Anliker to start the CAD_CAM_EDM_DRO
list on Yahoo which resulted in many more people
discovering and implementing EMC. In addition to
EMC, there were several other types of systems
demonstrated. Cardinal Engineering took pictures
of many of the systems and people displaying them
and intends to post them on the web. Hopefully
after Roland recovers from the trip we can find
the pictures on his company's web site at:

http://www.cardinaleng.com/

Sherline also announced at NAMES that they will
be selling a CNC system based on EMC. A demo
system was show at their booth.

While there was much discussion of EMC during the
show and at meals in the evening, there was a
special meeting session held on the Monday after
the show. This session or conference was open to
all who could make it and was intended to help
plan the future of EMC. While a similar effort was
attempted the previous year, this one was better
publicized, better attended, and more structured.

EMC is definitely alive and well. While NIST has
removed much of the public information about EMC
from their ISD web site, they are continuing to
research, develop and use motion control software.
EMC is only one of their uses for the RCS (Realtime
Control System) library. During the past year NIST
has been more active in other developments but has
not left EMC behind as some had rumored. Indeed, NIST
sent two of its primary developers, Fred Proctor
and Will Shackleford, as representatives to the
conference. While NIST remains committed to EMC and
has agreed in principal to respond to some of the
requests generated as a result of the conference,
no specific schedule or guarantee of completion are
implied.

EMC was, and still is a public domain effort and no
one entity directly controls the future of EMC.
This being said, the ad hoc group that met for the
conference had the goal of steering the future of
EMC. The summary of the conference below is just
that, a summary, not any sort of official document
issued by any controlling body.

--------

On the evening of Sunday, 27 April 2003, a sub-group
that had attended the NAMES show gathered to organize
an agenda for the conference on Monday. While many
issues were discussed the primary outcome was as
follows:

1) The EMC effort needs to be better presented to the
   public. Issues of insulating users from the sometimes
   very technical development issues created the desire
   to segregate the "experience" into developer and
   user camps with separate web sites and mailing lists.
   Some method of "governing" the EMC community is desired.

2) The documentation, both user and developer, needs to
   be improved. The user manual exists in multiple forms
   and locations and is not always complete or up to date.
   The organization of the source code makes becoming a
   developer a daunting task. If EMC is to move ahead in
   in user acceptance and code development, these issues
   need to be addressed.

3) New features (lathe mode, etc.) need to be added, bugs
   need to be documented and fixed, and structural changes
   to the code are needed to allow future development to
   be more modular and extensible.

4) Last, but not least, Matt Shaver was again "volunteered"
   to be the public figurehead for our ad hoc group.

The conference itself was convened at 8 AM Monday at the
Southgate Civic Center where the NAMES show had be held
over the weekend. This was an accomplishment in itself as
many participants had been up past midnight the night before
discussing EMC.

The attendees present, in alphabetical order by last name, were:

Douglas Albright        user, systems integrator
Paul Corner             user, developer, EMC evangelist
Craig Edwards           user, entrepreneur
Jon Elson               user, developer, entrepreneur, Pico Systems
John Farris             educator, Grand Valley State Univ.
John Gothberg           user
Ray Henry               user, developer, entrepreneur, EMC evangelist
Hugh Jack               educator, Grand Valley State Univ.
John Kasunich           user, developer
David Nichols           user, systems integrator, entrepreneur
Fred Proctor            developer, NIST
Hassan Rabe             user, student, Grand Valley State Univ.
Keith Rumley            user, machine designer, C++ programmer
Will Shackleford        developer, NIST
Matt Shaver             developer, systems integrator, EMC evangelist
Steve Stallings         user, entrepreneur, PMDX.com
David Volosky           user

The meeting was conducted without a fixed chairperson, but
was none the less reasonably organized and did cover the
desired agenda well. Because we were an ad hoc group and
no formal rules were set for the meeting, decisions were
arrived at by general consensus and no votes were taken.
Whenever the word "agreed" is used below to describe a
decision process, is refers to this type of a general
consensus and does not imply that everyone agreed.

Topic:

1) The EMC effort needs to be better presented to the
   public. Issues of insulating users from the sometimes
   very technical development issues created the desire
   to segregate the "experience" into developer and
   user camps with separate web sites and mailing lists.
   Some method of "governing" the EMC community is desired.

On the topic of governance and how EMC is presented in public,
the following were discussed and some items were resolved.

NIST is no longer providing user level information about EMC.
As mentioned before, their web pages have been updated to
reflect this. We may be able to restore some of the historic
information on another site, but it will have to cleaned up
to remove references to the old web site. NIST would also
like to be out of the list server business. The development
files now reside on the Source Forge site at:

http://sourceforge.net/projects/emc

This site also has a mailing list (emc-developers) that is
only sporadically used. There are also a set of forums,
one of which has the same name as the mailing list but
is separate and unused. It was agreed that the emc-developers
mailing list should become the developers mailing list of
choice. The archive of this list can be browsed on the
web site.  We still need to find a new home for a mailing
list for users. The web forums on Source Forge could be used,
but a mailing list would be preferred. Yahoo was suggested,
but was far from universally accepted. Whatever site does
come to exist, it was agreed that a safe, searchable archive
needs to exist. NIST has agreed to provide an archive of
the existing EMC-at-nist.gov list to be contributed to any
new list. Once this is accomplished the EMC-at-nist.gov list will
be converted to an autoresponder directing people to the new
developers and users lists.

*****************
Late breaking news..... it looks like we may be able to
host both the users list and the emc-developers list on the
Source Forge. To see what is if this proposed new list
has be set up, check out:

http://sourceforge.net/mail/?group_id=6744

Note: We are referring here to e-mail lists, not forums which
are web only and do not offer the user the opportunity to
keep his own private archive of the messages
*****************

The Source Forge site is already the "official" repository of
the EMC development code. NIST itself uses Source Forge for
their development of both EMC and the underlying RCS code.
NIST has also recognized the public's vested interest by
establishing an administrator role for one trusted member
of the EMC public community.

The Source Forge site needs to have a "last stable version
released" and a "current development state" sections. While
the BDI is the most popular method of getting a stable EMC
version to use in production, far too many people attempt
to download the current CVS version solely to get a usable
system to try to put into production.

LinuxCNC.org already exists as a web site to promote EMC.
It was generally agreed to keep it as the focus of promoting
EMC to the public. Rights to the LinuxCNC.org Internet domain
are owned by Dan Falck who had remained in the background
while allowing Steve Stallings to host the site, and for
Ray Henry and others to determine the content. He remains
interested and has recently become more involved despite
being unable to attend the conference. Other resources are
expected and still encouraged. LinuxCNC could also host
a user mailing list, but there is concern that rapid
growth could strain resources. Steve Stallings has agreed
to assist in making the LinuxCNC site more thorough and
user friendly and to investigate potential homes for the
user mailing list.

To restate - the general intent is to guide users to the
LinuxCNC.org web site and developers to the Source Forge
web site. Each site should reference the other. Large
bandwidth consumers, such as the Handbook and the current
stable version of the code, would be stored on the Source
Forge site even if the primary presentation of links is
on the LinuxCNC site.

The issues related to "governance" were discussed, but no
specific decisions were made. The possibility of setting up a
non-profit foundation was discussed but at this time no one is
going to directly pursue this. We remain an ad-hoc group.

All of the software code generated by NIST is, by US law, in
the public domain. The present body of EMC code contains other
code contributed by the public that is also in the public
domain. There is also some code (mostly user interface) and
some documentation (the EMC Handbook) that are covered under
GPL (General Public License). Concern was expressed that, under
the current situation, commercial organizations may utilize the
code and are not obligated to contribute any of their bug
fixes or improvements back to EMC. This will always be true
for portions generated by NIST (US law again), but public
contributions could be protected under the terms of the GPL.
For any license to be enforceable it must be issued by a
legal entity. This can be a person, a company, or a foundation.
There was a general discussion of whether the GPL was a good
idea and if so how could we accomplish this. After some brief
explanation of how the GPL functions, it was generally accepted
as a good idea. For more information on the GPL (General Public
License) see:

http://www.gnu.org/licenses/licenses.html

Since we are an ad hoc group, we cannot issue such a license.
Several people were concerned about the complexity of setting
up a corporation in order to be able to issue a license (and
perform other functions of interest to the group). It was
discussed and agreed to investigate working with the Free
Software Foundation. Situations such as this are usually
handled by assigning copyright ownership to them and then
the FSF licenses the software back to the public under the
GPL. The FSF is also in the best position to defend the
copyright since they designed the GPL and have access to some
of the best legal resources in the US (law school professors
from major universities), as well as on-staff attorneys. Matt
Shaver has taken on this responsibility.

Topic:

2) The documentation, both user and developer, needs to
   be improved. The user manual exists in multiple forms
   and locations and is not always complete or up to date.
   The organization of the source code makes becoming a
   developer a daunting task. If EMC is to move ahead in
   in user acceptance and code development, these issues
   need to be addressed.

The current state of the user documentation was briefly
discussed. While there was some concern about the accuracy
and completeness of this documentation, most of the concern
was about the divergent versions, where it should be hosted,
and what standard should be used to edit the master document.

Target formats of interest were HTML, Adobe PDF, and plain
text. The editing standard for the master document generated
quite a bit of discussion. The conflicting concerns were ease
of editing or contributing and ability to generate nice
looking output in the desired formats. It turns out that
is no trivial matter as anyone involved in publishing knows.
Normal word-processing formats can do reasonably well but
suffer from poor format conversion and operating system
limitations.

Typesetting formats offer the most promise for controlling
the appearance of the output. Recent efforts at updating
the "EMC Handbook" were focused on using the "TeX" language,
specifically the enhanced "LaTeX" version. While this
language originated on UNIX systems, it is available today
for most operating systems and is widely accepted in the
typesetting industry. No general consensus was reached, but
in the interest of continuity it is likely that LaTeX will
be the focus for the time being. Contributors who are
intimidated by LaTeX (or any formatting language for that
matter) could submit HTML, plain text, or even Word documents
to those who are comfortable using LaTeX. For more information
on LaTeX see:

http://www.tug.org/

Ray Henry has agreed to be the coordinator for getting the user
manuals updated.

The issue of documenting how to install EMC was not discussed
much. This is primarily because of the existence of the BDI
(brain dead install) CD ROMS. Paul Corner has done an excellent
job of creating these and has agreed to continue doing so. One
public relations issue was mentioned with respect to the BDI,
that is the possibility of removing the stigma of the name by
having an alternate definition of the acronym such as the
"Basic Distribution Installer".

Developer documentation issues revolve mostly around readability
of the code. Issues that were discussed include structural changes
to reduce the complexity of the compile options (IFDEF stuff),
making the modules smaller (no functions longer than two pages
of code), and formatting styles (indent control and use of
parentheses for readability even when not strictly required).
Many of these also imply changes to the structure of the code
as these issues are inseparable. NIST representatives agreed that
these things need to be done and will strive to make them happen.
Specifically they will address the IFDEF issue as soon as possible.

The other issues related to code documentation are the descriptions
of how the interpreter language is implemented, how the code modules
interrelate, and how messages and structures are defined. John
Kasunich
has agreed to work on the code documentation, focusing primarily on
EMCMOT and EMCIO, as well as providing an overview of the entire
system.  Additional volunteers are needed to document the EMCTASK
and GUI portions of the system.  NIST has agreed to review the
documentation for accuracy.

Topic:

3) New features (lathe mode, etc.) need to be added, bugs
   need to be documented and fixed, and structural changes
   to the code are needed to allow future development to
   be more modular and extensible.

The discussion of new features was extremely productive. Unlike
the previous year where we just discussed issues and set goals,
this year we were able to resolve how to reach some of the goals
and have work adopted by specific members.

In no particular order, we discussed these matters:

Bugs:

Interpreter flaws including inconsistency between EMC and other
industry accepted interpreters about the usage of "offsets",
and error reporting issues.

Motion control issues including feed rate control when using a
rotary axis, unintended motion of an uncommanded axis when
entering an MDI command before the previous move has completed,
the homing routine accumulating backlash distance each time homed.

General control issues including failure, sometimes, to save some
parameters on shutdown, and possible race conditions and ownership
conflicts issues with low speed I/O.

Structural improvements:

Separate initialization and cleanup code.

Break up large functions into more manageable modules.

Manage compile options for kernel version and real time libraries
by means of a wrapper function. Critical for cleanup of IFDEF stuff.

Allow for easier configuration without recompiling code, especially
with respect to I/O mapping and driver selection.

Make the interpreter more extensible by having G and M codes (other
than basic ones like G00, G01, G02, G03, M02, and M30 that are
clearly mandated to do one thing and for which all major industry
vendors agree on) mapped through a function dispatcher controlled by
initialization file values. Unused codes would map to error traps.
Complex codes, like drill cycles and offsets shifts, would become
user modifiable.

Support advanced features in interpreter, such as control over
angular axis modes of 0-360 degrees versus continuous rotation.

Change method of passing commands from motion planner to interpolators
to allow for control of maximum velocity and acceleration to be set
independently for each axis.

Enhance motion control to allow for S-curve acceleration techniques
and "jerk" control. This will require provision for the motion
planner to have more than the present fixed 4 points in the
interpolator queues. This may also provide a convenient point to
hook in for backing up short distances such as to clear a short
on a wire EDM machine.

Decouple the trajectory planner from servo loops so that trajectory
planner does not need to complete execution within one servo loop
cycle. This will be partly accommodated by the mid-level abstraction
discussed below.

Provide for bottom up servo and trajectory timing control rather than
the present top down control. Allow servo loops for different axes
to run at different rates. Still assume master timing from one source,
most likely the fastest servo loop.

Create a new hardware abstraction layer (HAL). This layer would be
the SOLE presentation of all hardware to other sections of the code.
As part of this, the non-real time I/O would now be routed through
the HAL and thus the real time system. This is consistent with Jon
Elson's work and is essential for other further development paths.

All control of bit mapping, signal polarity, and other hardware
specific items would be controlled below the HAL layer, thus a
signal such as "spindle" would always have the same sense (true
equals spindle on) above the HAL layer regardless of how the
spindle is actually turned on. Likewise a velocity would always
be represented the same way regardless of whether it actually
controlled an analog servo, a digital servo, or a stepper motor.
Stepper motors would always be required to simulate a servo to
the motion control software. Code below the HAL layer would be
modular and would have access to initialization file data to
configure actual signal polarities and other hardware specific
items. Multiple, dissimilar motor drivers could be installed.
Feedback and signal sensing would also pass through the HAL layer.

John Kasunich has agreed to be the point man for defining the
HAL. NIST and other developers will also be heavily involved.
The concept of a HAL is a major change to the EMC software and
we hope to generalize it enough to allow developers to accommodate
many types of radically different hardware without impacting the
main body of the code. Part of the concept may include having
the HAL extensible by the initialization file and possibly having
more than one HAL if required by abstraction at the mid-level layer,
discussed below.

Create a new mid-level layer of abstraction. Many possible names
for this abstraction layer were mentioned including task abstraction
layer (TAL), servo abstraction layer (SAL) and others, but nothing
seemed to stick. The concept is to abstract the presentation of
servo loops to the task planner. This is essential for allowing
the individualization of servo loops (independent update rates,
maximum velocity, and acceleration parameters) and for having
communication to servo loops via remote interfaces. While there
was no mention of any abstraction of I/O processes at this level
they would need to pass through this level and some abstraction
may be desirable. The possibility of remote interfaces may imply
the existence of multiple HALs.  No one has taken on responsibility
for the mid-level abstraction, but NIST intends to do something
because it is needed for some of their other projects. This is
likely to come after the HAL stuff is worked out.

New features:

Support for "electronic gearing" and related features needs to
be addressed. This is a prerequisite for lathe threading, rigid
tapping on mills, and support for jog wheels. Related issues
include rollover support for spindle encoders and how to handle
entry and exit of the slaved axis from electronic gearing mode.
NIST is interested in adding this support and, after the session
ended, they took advantage of the presence of knowledgeable users
to learn as much as possible about lathe threading before heading
back to the airport.

Lathes impose many other new requirements on the EMC software.
Tool offsets need to support two axes of offset (X and Z) as
well as a radius. Both the offsets must always be taken into
account as they are both in the "plane" of the cut unlike the
case for milling machines. The case of cutter compensation "on"
must also understand the tip radius of the tool. This differs
from in milling in that radius compensation is used to move the
tool further into the cut instead of moving back from it. This
is because the uncompensated cut assumes that the cutter tip
is square and always presents the full X and Z offset. When
a taper or arc is being cut, this is not true and in actual
fact the radius of the cutter tip causes the offset to be
reduced. Like a mill, the lathe still has "inside the cut"
and "outside the cut" issues, only the inside case actually
happens when the tool is truly inside the part such as when
boring a hole. There is no lathe equivalent to a pocket on a
mill. Most CNC lathes have built in procedures for measuring
and setting offsets. (Caution: The above paragraph is based
on my somewhat shaky understanding of CNC on a lathe.)

Other special cases related to lathes include provision for
constant surface speed for facing mode (requires spindle
speed control slaved to X axis travel), roughing cycles,
and threading cycles. Threading cycles can include multiple
passes, angle of progression on subsequent passes (90 degree,
29.5 degree, alternating 29.5 degree), finishing passes,
tapered threads, and multi-start threads. CNC lathes often
have extra axes for additional turrets, cutoff tools, and
power tailstocks.

Paul Corner has a CNC lathe which he has indicated that he
is willing to use to investigate lathe modes and to test
EMC lathe software as it is developed.

Canned cycles that are user configurable need to be added.
The case of the CNC lathe above demonstrates how important
this can become.

Subroutine support is industry standard and needs to be
added to EMC.

User macros are desirable and should be able to access
external routines by shell scripts, TCL, or other methods.
It is assumed that the external routines could interact
with EMC by NML messages or some other mechanism.

The interpreter could be made more modular or at least
selectively loaded based on initialization file data.

Motion issues related to rotary axes need to be addressed.
The current EMC code can generate unrealistically fast or
slow motions because the rotary axis feed rate is assumed to
be in degrees which have no direct relation to linear distance.
One quick fix for this problem is to cap the motion speed based
on the fastest moving axis and provide for a way to scale the
rotary axis feed rate factor to something useful. This can
work reasonably well for G00 but less so for G01 moves that
need actual feed rate control at the cutter tip. Another
way is to use the method developed by Rab Gordon. The math
of this method is explained in this message on the subject:

http://groups.yahoo.com/group/Master5/message/6218

A related description of the algorithm and assumptions about
radius can be found at:

http://groups.yahoo.com/group/Master5/message/6181

During the EMC conference another interesting approach to
rotary feed rate compensation was discussed. Unlike most
simple machine tool controls, EMC is capable of understanding
the geometry of the machine. This is called kinematics and
is not normally used for milling machines with rectilinear
motion. It was developed for robotics and more advanced
machine tools such as the Hexapod, but it could be used to
solve the rotary axis feed rate problem for a simple 4th axis.
It would be necessary to have data in the initialization file
to describe the setup of the rotary table. The math can be
somewhat complex (hyperbolic sines) but NIST believes that
some simplifying assumptions are possible for the case of
a single rotary axis parallel to one of the rectilinear
axes.

Another topic mentioned was enhancement of the homing routines.
This could include control of the sensing of the encoder index
signal and possibly user configurable back off algorithms.

One topic which was not covered, but came to me later was
to implement a watchdog signal that could be used to enable
external I/O. This might could be done below the HAL level,
but some thought should go into what a watchdog should really
be monitoring.

The last topic covered was testing strategies. No solution
to testing each EMC release on the wide variety of machines
was found. Testing compilation on the various software
environments was considered and it was generally decided
that future releases would be tested on the Linux 2.2 and
2.4 kernels with both RTL and RTAI real time libraries.
Support for Solaris and QNX would be retained but not
routinely tested. Support for VRTX and real time versions of
Windows NT would be dropped.

Most of the above features were of interest to NIST, but
no specific commitments were made. It is in the best interest
of the EMC user community to provide as much of our own effort
as possible. The documentation and structure improvements which
have already been committed to will go a long way to making
this easier to accomplish.

The meeting ended with a everyone tired but excited by the
progress made. It is expected that we will have another
similar meeting at NAMES 2004.

Best regards,
Steve Stallings
temporary secretary for the ad hoc EMC group