Error reading NML messages using Java

[Mark Bearden <mbearden-at-asinc.com>]

Hi, I would appreciate any likely problems or debugging tips anyone
can suggest with my problem, which I'll first state briefly, and then
give more details.

PROBLEM IN BRIEF:

I have a Java program attempting to read from a queued SHMEM buffer for
which the server is spawned by a master process that is a C++ program.
Both machines are running on the same RedHat 7.3 Linux host.

If I set the "neutral" buffer flag to 1 (do neutral encoding), then I
am getting this [abbreviated here, given in full below] error message
written to stdout by the C++ program, for each buffer read that I
attempt from within the Java program:

   + physmem.cc 391: PHYSMEM_HANDLE: Can't read 12 bytes at offset
   + 1275068458 from buffer of size 4064.
   + cms_in.cc 738: CMS:(consolectl) Error reading from global memory at
   + cms_in.cc:739

It so happens that the Java read() calls *do* seem to return the message
contents to the Java program.  However, the message is apparently not
cleared from the buffer.  If I call read() N times, I get the message
returned N times, even if I wrote it to the queued buffer only once.

On the other hand, if I set the "neutral" flag to 0 (don't encode),
then I get a different error.   The C++ process does not issue any
error messages, and the Java process reads an NMLmsg, but all the
fields in that message are empty (that is, having value 0).

Any ideas?

*******************
* MORE DETAILS
*******************

There's a third process in the picture, that I didn't mention above,
which is the one doing the writing into the buffer that the Java
process is reading.  That third process also runs on the same Linux
host and uses the C++ rcslib.  The version for both the C++ and
Java rcs libraries is 4.39, which I got this week from sourceforge.

Here's a picture:

     Proc: "cds"        Proc: "console_serv"      Proc: "console"
       WRITER  ------------>  MASTER  --------------> READER
        C++                     C++                    Java
                                 |
                                 |
                        SHMEM buffer: "consolectl"

and here are the NML config file contents for the same:

########################################################################
#  name       type  host      size  neut RPC# buf# max_proc key  ...
B  consolectl SHMEM 10.0.0.97 4096  0    0    13   10       1115      \
                                       TCP=5415  bsem=315 xdr   queue

#  name  buffer    type   host          ops srvr tmout master c# ...
P  cds   consolectl REMOTE 10.0.0.97      RW 0 4.000000 0   1 retry=2.0
P console_serv consolectl LOCAL 10.0.0.97 RW 2 4.000000 1   0 retry=2.0
P console  consolectl    REMOTE 10.0.0.97 RW 0 4.000000 0   3 retry=2.0
########################################################################

During my various debugging attempts I have reconfigured my system and
tried doing a read() on the consolectl buffer from within a C++-linked
process, with the writing process (cds) unchanged.  That read was
successful.  I've also been able in other experiments to successfully
write an NMLmsg using the Java program, and to correctly read that
message from a C++ process.

So in summary I can write to an NML buffer from my Java program, but
I can't read from a buffer.  Changing the "neutral" flag from 0 to 1
determines whether I observe an error at the C++ master/server program
(neutral = 1) or at the Java program (neutral = 0).

And here is the FULL text of the error message given by the C++ master
process each time I try a read() from the Java process:

   + physmem.cc 391: PHYSMEM_HANDLE: Can't read 12 bytes at offset
   + 1275068458 from buffer of size 4064.
   + cms_in.cc 738: CMS:(consolectl) Error reading from global memory at
   + cms_in.cc:739
   +
   + **********************************************************
   + * Current Directory = /home/mbearden/satssrc/sats/comm/bin
   + * -at-(#)$Info: RCS_LIBRARY_VERSION 4.39 Compiled on  Nov 13 2002 at
   + 16:59:29 for the linux_2_4_7_10smp platform with compiler version
   + 2.96   + 20000731 (Red Hat Linux 7.1 2.96-98) $ .
   +
   + **********************************************************
   + * BufferName = consolectl
   + * BufferType = 0
   + * ProcessName = console_serv
   + * Configuration File = ../../cds/bin/comm.nml
   + * CMS Status = -3 (CMS_INTERNAL_ACCESS_ERROR: An error occured
   + during an internal access function. )

Finally, I tried the same experiment with the buffer encoding
flag set to "disp" instead of "xdr".  I get yet a different
set of error behavior:

With "disp" and neutral=1, then the FIRST message I write to
buffer consolectl gets read correctly by the Java program,
and there are no error messages, but every subsequent read()
call returns null even if I have written additional messages
to the buffer (also written by cds process).

With "disp" and neutral=0, then I also get no complaints
from the buffer's master (console_serv), but at the Java
process I get this exception thrown when I call read on
the first message written:

NML read exception:
-------------
rcs.nml.NMLException: Format error buffer_name =consolectl config_file = 
comm.nml
         at rcs.nml.NMLConnection.readTCP(NMLConnection.java:1729)
         at rcs.nml.NMLConnection.read(NMLConnection.java:1152)

and again, all subsequent NMLConnection.read() calls return null,
even when multiple messages were written.

As if this weren't complicated enough already, I'll be happy to
provide more details if anyone wants to try to help me crack this.

Thanks!

Mark

========================================================================
Mark Bearden                          Applied Systems Intelligence, Inc.
Systems Engineer                      11660 Alpharetta Hwy., Suite 720
Phone: 770.518.4228 x237              Roswell, GA 30076 

   Fax: 770.619.0066                   http://www.asinc.com
========================================================================