Subject: [Vim-vms] Record format and attributes problems in VIM on VMS

From: Arpadffy Zoltan <[email-masked]>
Date: Wed, 4 Feb 2004 17:01:31 +0100

Frank,

you have shown that you have deep competence in VMS programming.
Hope you are willing to help us one more time with burning problem in VMS
system.

Vim uses block, binary write on C level (without sys$ calls), but sometimes
it has a very strange behaviour.
Actually this bug is thought to be solved somewhere in 6.0.x version, but I
got a new error report that I could verify that bug still exists and
requires competent and fast solution - because Vim will loose its
reliability.

Problem description:
RMS files can have different record format and attributes and a reliable
text editor should handle it equally safe. But seems for some reason
fab$c_var variable record format with fab$m_cr attribute is handled on
different way. We used a dirty solution of per line writing to solve this
anomaly, but recently one Vim user pointed out that   

If you read comments in FILEIO.C around line 3670 

#ifdef VMS
        /*
         * On VMS there is a problem: newlines get added when writing blocks
         * at a time. Fix it by writing a line at a time.
         * This is much slower!
         * Explanation: Vim can not handle, so far, variable record format.
         * With $analize/rms filename you can get the rms file structure,
and
         * if the Record format filed is variable, CR will be added after
         * every written buffer.  In other cases it works without this fix.
         * From other side read is about 5 times slower for "variable record
         * format" files.
         */

....you can see that I made an attempt to filter out fab$c_var , but after
this error report I made a hot fix by adding FAB$M_CR filter as well:

TOR_ZAY $ diff structs.h
************
File USRDSK:<ZAY.WORK.VIM62-181.SRC>STRUCTS.H;2
  898       char        b_fab_rfm;      /* Record format    */
  899       char        b_fab_rat;      /* Record attribute */
  900   #endif
******
File USRDSK:<ZAY.WORK.VIM62-181.SRC>STRUCTS.H;1
  898       char        b_fab_rfm;      /* Record format */
  899   #endif
************

 $ diff FILEIO.C;3 FILEIO.C;1
************
File USRDSK:<ZAY.WORK.VIM62-181.SRC>FILEIO.C;3
  454               curbuf->b_fab_rat = st.st_fab_rat;
  455   #endif
******
File USRDSK:<ZAY.WORK.VIM62-181.SRC>FILEIO.C;1
  454   #endif
************
************
File USRDSK:<ZAY.WORK.VIM62-181.SRC>FILEIO.C;3
 3674           if (buf->b_fab_rfm == FAB$C_VAR || buf->b_fab_rat ==
FAB$M_CR )
 3675           {
******
File USRDSK:<ZAY.WORK.VIM62-181.SRC>FILEIO.C;1
 3673           if (buf->b_fab_rfm == FAB$C_VAR)
 3674           {
************

Please note this is not delivered as a patch, and I (in the name of Vim
users community on VMS) would prefer to have a professional solution for
this problem and to have it well done. Therefore I ask competent HP
programmers to help us out.

RMS structures that I have detected problem are :   (below you can find
output from ANAL/RMS )

        File Organization: sequential
        Record Format: variable
        Record Attributes:  carriage-return
        Maximum Record Size: 0

These files get an CR after each record in block write

from other side:

        File Organization: sequential
        Record Format: stream
        Record Attributes:  carriage-return
        Maximum Record Size: 0

these files get a CR in per line write

Problem is easily reproduced:
1. take a longer textual file. Good example is a file filled with same
chars, 500 rows x 100 chars
2. set RMS file attributes
3. invoke vim filename
4. just :wq!  exit
5. run diff filename

result will be that files differ even if they should not and the difference
is an CR after each RMS record.

It is possible to try with different combinations in line
if (buf->b_fab_rfm == FAB$C_VAR || buf->b_fab_rat == FAB$M_CR )

... to see how it behaves... or ,that would be the best, to give a final
professional solution.

Thank you very much in advance.

Best regards, 
Z

---

From: Coen Engelbarts <[email-masked]>
Date: Thu, 05 Feb 2004 13:51:17 +0100

[email-masked] wrote on Wednesday 4 February 2004 17:02:

 > Vim uses block, binary write on C level (without sys$ calls), but
 > sometimes it has a very strange behaviour.

 >          * at a time. Fix it by writing a line at a time.

FWIW: Years ago I had a similar problem on MS-DOS. IIRC, MS-DOS added a 
LF (ASCII 10) after every CR (ASCII 13). On MS-DOS, you can solve this 
by opening the file explicitly in binary mode with:
     fopen(file, "wb");
i.e add a "b" to the mode string.

 From the Linux man page for fopen():

        The mode string can also include the letter  ``b''  either
        as  a last character or as a character between the charac
        ters in any of the two-character strings described  above.
        This  is  strictly for compatibility with ANSI X3.159-1989
        (``ANSI C'') and has no effect; the ``b''  is  ignored  on
        all  POSIX  conforming  systems,  including Linux.  (Other
        systems may treat text files and binary files differently,
        and adding the ``b'' may be a good idea if you do I/O to a
        binary file and expect that your program may be ported  to
        non-Unix environments.)

I doubt that this solution will work on VMS (which AFAIK was the first 
POSIX compliant OS, even before any Unix version), but I thought I'd 
mention it just in case.

 >          * Explanation: Vim can not handle, so far, variable record

I suppose that the proper solution would be to add native RMS file 
handling to Vim, but I'm not an RMS-expert either.
All I know is that DEC/Compaq/HP-C supports extensions to fopen() etc., 
which allow you to specify RMS-attributes.

Regards,

Coen

As an aside: My MS-DOS program at the time was a simulation/test tool 
for a big egg sorting machine, and the "binary" file contained the 
weights of eggs in the simulated machine. It was quite weird to see the 
program create (dare I say "lay"?) eggs at seemingly random places, 
until I realised that they all weighed _exacly_ 10 grams and were 
_always_ preceded by a 13-gram egg...
Dûh!
:-}