GEMPAK Manual | Programs | Parameters

     PRECSN

     PRECSN is the binary or decimal precision to be preserved in the
     packing of the gridded data.

     Binary precision is entered as a B/n, where B indicates binary
     precision and n is an integer indicating the nearest power of 2
     to which to pack the data.  Power of 10 scaling of the data is
     NOT done when binary precision is specified.  The following
     table relates values of n to the precision retained.

             n             Binary Precision Description

            -4       Data values are rounded to the nearest 1/16.
            -3       Data values are rounded to the nearest 1/8.
            -2       Data values are rounded to the nearest 1/4.
            -1       Data values are rounded to the nearest 1/2.
             0       Data values are rounded to the nearest 1.
             1       Data values are rounded to the nearest 2.
             2       Data values are rounded to the nearest 4.
             3       Data values are rounded to the nearest 8.
             4       Data values are rounded to the nearest 16.

     Decimal precision is entered as D/r, where D indicates decimal
     precision and r is a positive floating point number indicating
     the number of decimal significant digits to preserve.  The
     fractional part of r is retained in the calculation of the
     number of packing bits required and may be used to enhance
     precision in cases where binary precision might require too
     many bits.  The necessary power of 10 scaling is determined
     automatically.