/numpy/f2py/crackfortran.py

Large files files are truncated, but you can click here to view the full file

#!/usr/bin/env python
"""
crackfortran --- read fortran (77,90) code and extract declaration information.
    Usage is explained in the comment block below.

Copyright 1999-2004 Pearu Peterson all rights reserved,
Pearu Peterson <pearu@ioc.ee>
Permission to use, modify, and distribute this software is given under the
terms of the NumPy License.

NO WARRANTY IS EXPRESSED OR IMPLIED.  USE AT YOUR OWN RISK.
$Date: 2005/09/27 07:13:49 $
Pearu Peterson
"""
__version__ = "$Revision: 1.177 $"[10:-1]

import __version__
f2py_version = __version__.version

"""
 Usage of crackfortran:
 ======================
 Command line keys: -quiet,-verbose,-fix,-f77,-f90,-show,-h <pyffilename>
                    -m <module name for f77 routines>,--ignore-contains
 Functions: crackfortran, crack2fortran
 The following Fortran statements/constructions are supported
 (or will be if needed):
    block data,byte,call,character,common,complex,contains,data,
    dimension,double complex,double precision,end,external,function,
    implicit,integer,intent,interface,intrinsic,
    logical,module,optional,parameter,private,public,
    program,real,(sequence?),subroutine,type,use,virtual,
    include,pythonmodule
 Note: 'virtual' is mapped to 'dimension'.
 Note: 'implicit integer (z) static (z)' is 'implicit static (z)' (this is minor bug).
 Note: code after 'contains' will be ignored until its scope ends.
 Note: 'common' statement is extended: dimensions are moved to variable definitions
 Note: f2py directive: <commentchar>f2py<line> is read as <line>
 Note: pythonmodule is introduced to represent Python module

 Usage:
   `postlist=crackfortran(files,funcs)`
   `postlist` contains declaration information read from the list of files `files`.
   `crack2fortran(postlist)` returns a fortran code to be saved to pyf-file

   `postlist` has the following structure:
  *** it is a list of dictionaries containing `blocks':
      B = {'block','body','vars','parent_block'[,'name','prefix','args','result',
           'implicit','externals','interfaced','common','sortvars',
           'commonvars','note']}
      B['block'] = 'interface' | 'function' | 'subroutine' | 'module' |
                   'program' | 'block data' | 'type' | 'pythonmodule'
      B['body'] --- list containing `subblocks' with the same structure as `blocks'
      B['parent_block'] --- dictionary of a parent block:
                              C['body'][<index>]['parent_block'] is C
      B['vars'] --- dictionary of variable definitions
      B['sortvars'] --- dictionary of variable definitions sorted by dependence (independent first)
      B['name'] --- name of the block (not if B['block']=='interface')
      B['prefix'] --- prefix string (only if B['block']=='function')
      B['args'] --- list of argument names if B['block']== 'function' | 'subroutine'
      B['result'] --- name of the return value (only if B['block']=='function')
      B['implicit'] --- dictionary {'a':<variable definition>,'b':...} | None
      B['externals'] --- list of variables being external
      B['interfaced'] --- list of variables being external and defined
      B['common'] --- dictionary of common blocks (list of objects)
      B['commonvars'] --- list of variables used in common blocks (dimensions are moved to variable definitions)
      B['from'] --- string showing the 'parents' of the current block
      B['use'] --- dictionary of modules used in current block:
          {<modulename>:{['only':<0|1>],['map':{<local_name1>:<use_name1>,...}]}}
      B['note'] --- list of LaTeX comments on the block
      B['f2pyenhancements'] --- optional dictionary
           {'threadsafe':'','fortranname':<name>,
            'callstatement':<C-expr>|<multi-line block>,
            'callprotoargument':<C-expr-list>,
            'usercode':<multi-line block>|<list of multi-line blocks>,
            'pymethoddef:<multi-line block>'
            }
      B['entry'] --- dictionary {entryname:argslist,..}
      B['varnames'] --- list of variable names given in the order of reading the
                        Fortran code, useful for derived types.
  *** Variable definition is a dictionary
      D = B['vars'][<variable name>] =
      {'typespec'[,'attrspec','kindselector','charselector','=','typename']}
      D['typespec'] = 'byte' | 'character' | 'complex' | 'double complex' |
                      'double precision' | 'integer' | 'logical' | 'real' | 'type'
      D['attrspec'] --- list of attributes (e.g. 'dimension(<arrayspec>)',
                        'external','intent(in|out|inout|hide|c|callback|cache|aligned4|aligned8|aligned16)',
                        'optional','required', etc)
      K = D['kindselector'] = {['*','kind']} (only if D['typespec'] =
                          'complex' | 'integer' | 'logical' | 'real' )
      C = D['charselector'] = {['*','len','kind']}
                              (only if D['typespec']=='character')
      D['='] --- initialization expression string
      D['typename'] --- name of the type if D['typespec']=='type'
      D['dimension'] --- list of dimension bounds
      D['intent'] --- list of intent specifications
      D['depend'] --- list of variable names on which current variable depends on
      D['check'] --- list of C-expressions; if C-expr returns zero, exception is raised
      D['note'] --- list of LaTeX comments on the variable
  *** Meaning of kind/char selectors (few examples):
      D['typespec>']*K['*']
      D['typespec'](kind=K['kind'])
      character*C['*']
      character(len=C['len'],kind=C['kind'])
      (see also fortran type declaration statement formats below)

 Fortran 90 type declaration statement format (F77 is subset of F90)
====================================================================
 (Main source: IBM XL Fortran 5.1 Language Reference Manual)
 type declaration = <typespec> [[<attrspec>]::] <entitydecl>
 <typespec> = byte                          |
              character[<charselector>]     |
              complex[<kindselector>]       |
              double complex                |
              double precision              |
              integer[<kindselector>]       |
              logical[<kindselector>]       |
              real[<kindselector>]          |
              type(<typename>)
 <charselector> = * <charlen>               |
              ([len=]<len>[,[kind=]<kind>]) |
              (kind=<kind>[,len=<len>])
 <kindselector> = * <intlen>                |
              ([kind=]<kind>)
 <attrspec> = comma separated list of attributes.
              Only the following attributes are used in
              building up the interface:
                 external
                 (parameter --- affects '=' key)
                 optional
                 intent
              Other attributes are ignored.
 <intentspec> = in | out | inout
 <arrayspec> = comma separated list of dimension bounds.
 <entitydecl> = <name> [[*<charlen>][(<arrayspec>)] | [(<arrayspec>)]*<charlen>]
                       [/<init_expr>/ | =<init_expr>] [,<entitydecl>]

 In addition, the following attributes are used: check,depend,note

 TODO:
     * Apply 'parameter' attribute (e.g. 'integer parameter :: i=2' 'real x(i)'
                                    -> 'real x(2)')
     The above may be solved by creating appropriate preprocessor program, for example.
"""
#
import sys
import string
import fileinput
import re
import pprint
import os
import copy
from auxfuncs import *

# Global flags:
strictf77=1          # Ignore `!' comments unless line[0]=='!'
sourcecodeform='fix' # 'fix','free'
quiet=0              # Be verbose if 0 (Obsolete: not used any more)
verbose=1            # Be quiet if 0, extra verbose if > 1.
tabchar=4*' '
pyffilename=''
f77modulename=''
skipemptyends=0      # for old F77 programs without 'program' statement
ignorecontains=1
dolowercase=1
debug=[]
## do_analyze = 1

###### global variables

## use reload(crackfortran) to reset these variables

groupcounter=0
grouplist={groupcounter:[]}
neededmodule=-1
expectbegin=1
skipblocksuntil=-1
usermodules=[]
f90modulevars={}
gotnextfile=1
filepositiontext=''
currentfilename=''
skipfunctions=[]
skipfuncs=[]
onlyfuncs=[]
include_paths=[]
previous_context = None

###### Some helper functions
def show(o,f=0):pprint.pprint(o)
errmess=sys.stderr.write
def outmess(line,flag=1):
    global filepositiontext
    if not verbose: return
    if not quiet:
        if flag:sys.stdout.write(filepositiontext)
        sys.stdout.write(line)
re._MAXCACHE=50
defaultimplicitrules={}
for c in "abcdefghopqrstuvwxyz$_": defaultimplicitrules[c]={'typespec':'real'}
for c in "ijklmn": defaultimplicitrules[c]={'typespec':'integer'}
del c
badnames={}
invbadnames={}
for n in ['int','double','float','char','short','long','void','case','while',
          'return','signed','unsigned','if','for','typedef','sizeof','union',
          'struct','static','register','new','break','do','goto','switch',
          'continue','else','inline','extern','delete','const','auto',
          'len','rank','shape','index','slen','size','_i',
          'max', 'min',
          'flen','fshape',
          'string','complex_double','float_double','stdin','stderr','stdout',
          'type','default']:
    badnames[n]=n+'_bn'
    invbadnames[n+'_bn']=n
def rmbadname1(name):
    if name in badnames:
        errmess('rmbadname1: Replacing "%s" with "%s".\n'%(name,badnames[name]))
        return badnames[name]
    return name
def rmbadname(names): return map(rmbadname1,names)

def undo_rmbadname1(name):
    if name in invbadnames:
        errmess('undo_rmbadname1: Replacing "%s" with "%s".\n'\
                %(name,invbadnames[name]))
        return invbadnames[name]
    return name
def undo_rmbadname(names): return map(undo_rmbadname1,names)

def getextension(name):
    i=name.rfind('.')
    if i==-1: return ''
    if '\\' in name[i:]: return ''
    if '/' in name[i:]: return ''
    return name[i+1:]

is_f_file = re.compile(r'.*[.](for|ftn|f77|f)\Z',re.I).match
_has_f_header = re.compile(r'-[*]-\s*fortran\s*-[*]-',re.I).search
_has_f90_header = re.compile(r'-[*]-\s*f90\s*-[*]-',re.I).search
_has_fix_header = re.compile(r'-[*]-\s*fix\s*-[*]-',re.I).search
_free_f90_start = re.compile(r'[^c*]\s*[^\s\d\t]',re.I).match
def is_free_format(file):
    """Check if file is in free format Fortran."""
    # f90 allows both fixed and free format, assuming fixed unless
    # signs of free format are detected.
    result = 0
    f = open(file,'r')
    line = f.readline()
    n = 15 # the number of non-comment lines to scan for hints
    if _has_f_header(line):
        n = 0
    elif _has_f90_header(line):
        n = 0
        result = 1
    while n>0 and line:
        if line[0]!='!' and line.strip():
            n -= 1
            if (line[0]!='\t' and _free_f90_start(line[:5])) or line[-2:-1]=='&':
                result = 1
                break
        line = f.readline()
    f.close()
    return result


####### Read fortran (77,90) code
def readfortrancode(ffile,dowithline=show,istop=1):
    """
    Read fortran codes from files and
     1) Get rid of comments, line continuations, and empty lines; lower cases.
     2) Call dowithline(line) on every line.
     3) Recursively call itself when statement \"include '<filename>'\" is met.
    """
    global gotnextfile,filepositiontext,currentfilename,sourcecodeform,strictf77,\
           beginpattern,quiet,verbose,dolowercase,include_paths
    if not istop:
        saveglobals=gotnextfile,filepositiontext,currentfilename,sourcecodeform,strictf77,\
           beginpattern,quiet,verbose,dolowercase
    if ffile==[]: return
    localdolowercase = dolowercase
    cont=0
    finalline=''
    ll=''
    commentline=re.compile(r'(?P<line>([^"]*"[^"]*"[^"!]*|[^\']*\'[^\']*\'[^\'!]*|[^!]*))!{1}(?P<rest>.*)')
    includeline=re.compile(r'\s*include\s*(\'|")(?P<name>[^\'"]*)(\'|")',re.I)
    cont1=re.compile(r'(?P<line>.*)&\s*\Z')
    cont2=re.compile(r'(\s*&|)(?P<line>.*)')
    mline_mark = re.compile(r".*?'''")
    if istop: dowithline('',-1)
    ll,l1='',''
    spacedigits=[' ']+map(str,range(10))
    filepositiontext=''
    fin=fileinput.FileInput(ffile)
    while 1:
        l=fin.readline()
        if not l: break
        if fin.isfirstline():
            filepositiontext=''
            currentfilename=fin.filename()
            gotnextfile=1
            l1=l
            strictf77=0
            sourcecodeform='fix'
            ext = os.path.splitext(currentfilename)[1]
            if is_f_file(currentfilename) and \
                   not (_has_f90_header(l) or _has_fix_header(l)):
                strictf77=1
            elif is_free_format(currentfilename) and not _has_fix_header(l):
                sourcecodeform='free'
            if strictf77: beginpattern=beginpattern77
            else: beginpattern=beginpattern90
            outmess('\tReading file %s (format:%s%s)\n'\
                    %(`currentfilename`,sourcecodeform,
                      strictf77 and ',strict' or ''))

        l=l.expandtabs().replace('\xa0',' ')
        while not l=='':                       # Get rid of newline characters
            if l[-1] not in "\n\r\f": break
            l=l[:-1]
        if not strictf77:
            r=commentline.match(l)
            if r:
                l=r.group('line')+' ' # Strip comments starting with `!'
                rl=r.group('rest')
                if rl[:4].lower()=='f2py': # f2py directive
                    l = l + 4*' '
                    r=commentline.match(rl[4:])
                    if r: l=l+r.group('line')
                    else: l = l + rl[4:]
        if l.strip()=='': # Skip empty line
            cont=0
            continue
        if sourcecodeform=='fix':
            if l[0] in ['*','c','!','C','#']:
                if l[1:5].lower()=='f2py': # f2py directive
                    l='     '+l[5:]
                else: # Skip comment line
                    cont=0
                    continue
            elif strictf77:
                if len(l)>72: l=l[:72]
            if not (l[0] in spacedigits):
                raise Exception('readfortrancode: Found non-(space,digit) char '
                                'in the first column.\n\tAre you sure that '
                                'this code is in fix form?\n\tline=%s' % `l`)

            if (not cont or strictf77) and (len(l)>5 and not l[5]==' '):
                # Continuation of a previous line
                ll=ll+l[6:]
                finalline=''
                origfinalline=''
            else:
                if not strictf77:
                    # F90 continuation
                    r=cont1.match(l)
                    if r: l=r.group('line') # Continuation follows ..
                    if cont:
                        ll=ll+cont2.match(l).group('line')
                        finalline=''
                        origfinalline=''
                    else:
                        l='     '+l[5:] # clean up line beginning from possible digits.
                        if localdolowercase: finalline=ll.lower()
                        else: finalline=ll
                        origfinalline=ll
                        ll=l
                    cont=(r is not None)
                else:
                    l='     '+l[5:] # clean up line beginning from possible digits.
                    if localdolowercase: finalline=ll.lower()
                    else: finalline=ll
                    origfinalline =ll
                    ll=l

        elif sourcecodeform=='free':
            if not cont and ext=='.pyf' and mline_mark.match(l):
                l = l + '\n'
                while 1:
                    lc = fin.readline()
                    if not lc:
                        errmess('Unexpected end of file when reading multiline\n')
                        break
                    l = l + lc
                    if mline_mark.match(lc):
                        break
                l = l.rstrip()
            r=cont1.match(l)
            if r: l=r.group('line') # Continuation follows ..
            if cont:
                ll=ll+cont2.match(l).group('line')
                finalline=''
                origfinalline=''
            else:
                if localdolowercase: finalline=ll.lower()
                else: finalline=ll
                origfinalline =ll
                ll=l
            cont=(r is not None)
        else:
            raise ValueError,"Flag sourcecodeform must be either 'fix' or 'free': %s"%`sourcecodeform`
        filepositiontext='Line #%d in %s:"%s"\n\t' % (fin.filelineno()-1,currentfilename,l1)
        m=includeline.match(origfinalline)
        if m:
            fn=m.group('name')
            if os.path.isfile(fn):
                readfortrancode(fn,dowithline=dowithline,istop=0)
            else:
                include_dirs = [os.path.dirname(currentfilename)] + include_paths
                foundfile = 0
                for inc_dir in include_dirs:
                    fn1 = os.path.join(inc_dir,fn)
                    if os.path.isfile(fn1):
                        foundfile = 1
                        readfortrancode(fn1,dowithline=dowithline,istop=0)
                        break
                if not foundfile:
                    outmess('readfortrancode: could not find include file %s. Ignoring.\n'%(`fn`))
        else:
            dowithline(finalline)
        l1=ll
    if localdolowercase:
        finalline=ll.lower()
    else: finalline=ll
    origfinalline = ll
    filepositiontext='Line #%d in %s:"%s"\n\t' % (fin.filelineno()-1,currentfilename,l1)
    m=includeline.match(origfinalline)
    if m:
        fn=m.group('name')
        fn1=os.path.join(os.path.dirname(currentfilename),fn)
        if os.path.isfile(fn):
            readfortrancode(fn,dowithline=dowithline,istop=0)
        elif os.path.isfile(fn1):
            readfortrancode(fn1,dowithline=dowithline,istop=0)
        else:
            outmess('readfortrancode: could not find include file %s. Ignoring.\n'%(`fn`))
    else:
        dowithline(finalline)
    filepositiontext=''
    fin.close()
    if istop: dowithline('',1)
    else:
        gotnextfile,filepositiontext,currentfilename,sourcecodeform,strictf77,\
           beginpattern,quiet,verbose,dolowercase=saveglobals

########### Crack line
beforethisafter=r'\s*(?P<before>%s(?=\s*(\b(%s)\b)))'+ \
                          r'\s*(?P<this>(\b(%s)\b))'+ \
                          r'\s*(?P<after>%s)\s*\Z'
##
fortrantypes='character|logical|integer|real|complex|double\s*(precision\s*(complex|)|complex)|type(?=\s*\([\w\s,=(*)]*\))|byte'
typespattern=re.compile(beforethisafter%('',fortrantypes,fortrantypes,'.*'),re.I),'type'
typespattern4implicit=re.compile(beforethisafter%('',fortrantypes+'|static|automatic|undefined',fortrantypes+'|static|automatic|undefined','.*'),re.I)
#
functionpattern=re.compile(beforethisafter%('([a-z]+[\w\s(=*+-/)]*?|)','function','function','.*'),re.I),'begin'
subroutinepattern=re.compile(beforethisafter%('[a-z\s]*?','subroutine','subroutine','.*'),re.I),'begin'
#modulepattern=re.compile(beforethisafter%('[a-z\s]*?','module','module','.*'),re.I),'begin'
#
groupbegins77=r'program|block\s*data'
beginpattern77=re.compile(beforethisafter%('',groupbegins77,groupbegins77,'.*'),re.I),'begin'
groupbegins90=groupbegins77+r'|module(?!\s*procedure)|python\s*module|interface|type(?!\s*\()'
beginpattern90=re.compile(beforethisafter%('',groupbegins90,groupbegins90,'.*'),re.I),'begin'
groupends=r'end|endprogram|endblockdata|endmodule|endpythonmodule|endinterface'
endpattern=re.compile(beforethisafter%('',groupends,groupends,'[\w\s]*'),re.I),'end'
#endifs='end\s*(if|do|where|select|while|forall)'
endifs='(end\s*(if|do|where|select|while|forall))|(module\s*procedure)'
endifpattern=re.compile(beforethisafter%('[\w]*?',endifs,endifs,'[\w\s]*'),re.I),'endif'
#
implicitpattern=re.compile(beforethisafter%('','implicit','implicit','.*'),re.I),'implicit'
dimensionpattern=re.compile(beforethisafter%('','dimension|virtual','dimension|virtual','.*'),re.I),'dimension'
externalpattern=re.compile(beforethisafter%('','external','external','.*'),re.I),'external'
optionalpattern=re.compile(beforethisafter%('','optional','optional','.*'),re.I),'optional'
requiredpattern=re.compile(beforethisafter%('','required','required','.*'),re.I),'required'
publicpattern=re.compile(beforethisafter%('','public','public','.*'),re.I),'public'
privatepattern=re.compile(beforethisafter%('','private','private','.*'),re.I),'private'
intrisicpattern=re.compile(beforethisafter%('','intrisic','intrisic','.*'),re.I),'intrisic'
intentpattern=re.compile(beforethisafter%('','intent|depend|note|check','intent|depend|note|check','\s*\(.*?\).*'),re.I),'intent'
parameterpattern=re.compile(beforethisafter%('','parameter','parameter','\s*\(.*'),re.I),'parameter'
datapattern=re.compile(beforethisafter%('','data','data','.*'),re.I),'data'
callpattern=re.compile(beforethisafter%('','call','call','.*'),re.I),'call'
entrypattern=re.compile(beforethisafter%('','entry','entry','.*'),re.I),'entry'
callfunpattern=re.compile(beforethisafter%('','callfun','callfun','.*'),re.I),'callfun'
commonpattern=re.compile(beforethisafter%('','common','common','.*'),re.I),'common'
usepattern=re.compile(beforethisafter%('','use','use','.*'),re.I),'use'
containspattern=re.compile(beforethisafter%('','contains','contains',''),re.I),'contains'
formatpattern=re.compile(beforethisafter%('','format','format','.*'),re.I),'format'
## Non-fortran and f2py-specific statements
f2pyenhancementspattern=re.compile(beforethisafter%('','threadsafe|fortranname|callstatement|callprotoargument|usercode|pymethoddef','threadsafe|fortranname|callstatement|callprotoargument|usercode|pymethoddef','.*'),re.I|re.S),'f2pyenhancements'
multilinepattern = re.compile(r"\s*(?P<before>''')(?P<this>.*?)(?P<after>''')\s*\Z",re.S),'multiline'
##

def _simplifyargs(argsline):
    a = []
    for n in markoutercomma(argsline).split('@,@'):
        for r in '(),':
            n = n.replace(r,'_')
        a.append(n)
    return ','.join(a)

crackline_re_1 = re.compile(r'\s*(?P<result>\b[a-z]+[\w]*\b)\s*[=].*',re.I)
def crackline(line,reset=0):
    """
    reset=-1  --- initialize
    reset=0   --- crack the line
    reset=1   --- final check if mismatch of blocks occured

    Cracked data is saved in grouplist[0].
    """
    global beginpattern,groupcounter,groupname,groupcache,grouplist,gotnextfile,\
           filepositiontext,currentfilename,neededmodule,expectbegin,skipblocksuntil,\
           skipemptyends,previous_context
    if ';' in line and not (f2pyenhancementspattern[0].match(line) or
                            multilinepattern[0].match(line)):
        for l in line.split(';'):
            assert reset==0,`reset` # XXX: non-zero reset values need testing
            crackline(l,reset)
        return
    if reset<0:
        groupcounter=0
        groupname={groupcounter:''}
        groupcache={groupcounter:{}}
        grouplist={groupcounter:[]}
        groupcache[groupcounter]['body']=[]
        groupcache[groupcounter]['vars']={}
        groupcache[groupcounter]['block']=''
        groupcache[groupcounter]['name']=''
        neededmodule=-1
        skipblocksuntil=-1
        return
    if reset>0:
        fl=0
        if f77modulename and neededmodule==groupcounter: fl=2
        while groupcounter>fl:
            outmess('crackline: groupcounter=%s groupname=%s\n'%(`groupcounter`,`groupname`))
            outmess('crackline: Mismatch of blocks encountered. Trying to fix it by assuming "end" statement.\n')
            grouplist[groupcounter-1].append(groupcache[groupcounter])
            grouplist[groupcounter-1][-1]['body']=grouplist[groupcounter]
            del grouplist[groupcounter]
            groupcounter=groupcounter-1
        if f77modulename and neededmodule==groupcounter:
            grouplist[groupcounter-1].append(groupcache[groupcounter])
            grouplist[groupcounter-1][-1]['body']=grouplist[groupcounter]
            del grouplist[groupcounter]
            groupcounter=groupcounter-1 # end interface
            grouplist[groupcounter-1].append(groupcache[groupcounter])
            grouplist[groupcounter-1][-1]['body']=grouplist[groupcounter]
            del grouplist[groupcounter]
            groupcounter=groupcounter-1 # end module
            neededmodule=-1
        return
    if line=='': return
    flag=0
    for pat in [dimensionpattern,externalpattern,intentpattern,optionalpattern,
                requiredpattern,
                parameterpattern,datapattern,publicpattern,privatepattern,
                intrisicpattern,
                endifpattern,endpattern,
                formatpattern,
                beginpattern,functionpattern,subroutinepattern,
                implicitpattern,typespattern,commonpattern,
                callpattern,usepattern,containspattern,
                entrypattern,
                f2pyenhancementspattern,
                multilinepattern
                ]:
        m = pat[0].match(line)
        if m:
            break
        flag=flag+1
    if not m:
        re_1 = crackline_re_1
        if 0<=skipblocksuntil<=groupcounter:return
        if 'externals' in groupcache[groupcounter]:
            for name in groupcache[groupcounter]['externals']:
                if name in invbadnames:
                    name=invbadnames[name]
                if 'interfaced' in groupcache[groupcounter] and name in groupcache[groupcounter]['interfaced']:
                    continue
                m1=re.match(r'(?P<before>[^"]*)\b%s\b\s*@\(@(?P<args>[^@]*)@\)@.*\Z'%name,markouterparen(line),re.I)
                if m1:
                    m2 = re_1.match(m1.group('before'))
                    a = _simplifyargs(m1.group('args'))
                    if m2:
                        line='callfun %s(%s) result (%s)'%(name,a,m2.group('result'))
                    else: line='callfun %s(%s)'%(name,a)
                    m = callfunpattern[0].match(line)
                    if not m:
                        outmess('crackline: could not resolve function call for line=%s.\n'%`line`)
                        return
                    analyzeline(m,'callfun',line)
                    return
        if verbose>1:
            previous_context = None
            outmess('crackline:%d: No pattern for line\n'%(groupcounter))
        return
    elif pat[1]=='end':
        if 0<=skipblocksuntil<groupcounter:
            groupcounter=groupcounter-1
            if skipblocksuntil<=groupcounter: return
        if groupcounter<=0:
            raise Exception('crackline: groupcounter(=%s) is nonpositive. '
                            'Check the blocks.' \
                            % (groupcounter))
        m1 = beginpattern[0].match((line))
        if (m1) and (not m1.group('this')==groupname[groupcounter]):
            raise Exception('crackline: End group %s does not match with '
                            'previous Begin group %s\n\t%s' % \
                            (`m1.group('this')`, `groupname[groupcounter]`,
                             filepositiontext)
                            )
        if skipblocksuntil==groupcounter:
            skipblocksuntil=-1
        grouplist[groupcounter-1].append(groupcache[groupcounter])
        grouplist[groupcounter-1][-1]['body']=grouplist[groupcounter]
        del grouplist[groupcounter]
        groupcounter=groupcounter-1
        if not skipemptyends:
            expectbegin=1
    elif pat[1] == 'begin':
        if 0<=skipblocksuntil<=groupcounter:
            groupcounter=groupcounter+1
            return
        gotnextfile=0
        analyzeline(m,pat[1],line)
        expectbegin=0
    elif pat[1]=='endif':
        pass
    elif pat[1]=='contains':
        if ignorecontains: return
        if 0<=skipblocksuntil<=groupcounter: return
        skipblocksuntil=groupcounter
    else:
        if 0<=skipblocksuntil<=groupcounter:return
        analyzeline(m,pat[1],line)

def markouterparen(line):
    l='';f=0
    for c in line:
        if c=='(':
            f=f+1
            if f==1: l=l+'@(@'; continue
        elif c==')':
            f=f-1
            if f==0: l=l+'@)@'; continue
        l=l+c
    return l
def markoutercomma(line,comma=','):
    l='';f=0
    cc=''
    for c in line:
        if (not cc or cc==')') and c=='(':
            f=f+1
            cc = ')'
        elif not cc and c=='\'' and (not l or l[-1]!='\\'):
            f=f+1
            cc = '\''
        elif c==cc:
            f=f-1
            if f==0:
                cc=''
        elif c==comma and f==0:
            l=l+'@'+comma+'@'
            continue
        l=l+c
    assert not f,`f,line,l,cc`
    return l
def unmarkouterparen(line):
    r = line.replace('@(@','(').replace('@)@',')')
    return r
def appenddecl(decl,decl2,force=1):
    if not decl: decl={}
    if not decl2: return decl
    if decl is decl2: return decl
    for k in decl2.keys():
        if k=='typespec':
            if force or k not in decl:
                decl[k]=decl2[k]
        elif k=='attrspec':
            for l in decl2[k]:
                decl=setattrspec(decl,l,force)
        elif k=='kindselector':
            decl=setkindselector(decl,decl2[k],force)
        elif k=='charselector':
            decl=setcharselector(decl,decl2[k],force)
        elif k in ['=','typename']:
            if force or k not in decl:
                decl[k]=decl2[k]
        elif k=='note':
            pass
        elif k in ['intent','check','dimension','optional','required']:
            errmess('appenddecl: "%s" not implemented.\n'%k)
        else:
            raise Exception('appenddecl: Unknown variable definition key:' + \
                            str(k))
    return decl

selectpattern=re.compile(r'\s*(?P<this>(@\(@.*?@\)@|[*][\d*]+|[*]\s*@\(@.*?@\)@|))(?P<after>.*)\Z',re.I)
nameargspattern=re.compile(r'\s*(?P<name>\b[\w$]+\b)\s*(@\(@\s*(?P<args>[\w\s,]*)\s*@\)@|)\s*(result(\s*@\(@\s*(?P<result>\b[\w$]+\b)\s*@\)@|))*\s*\Z',re.I)
callnameargspattern=re.compile(r'\s*(?P<name>\b[\w$]+\b)\s*@\(@\s*(?P<args>.*)\s*@\)@\s*\Z',re.I)
real16pattern = re.compile(r'([-+]?(?:\d+(?:\.\d*)?|\d*\.\d+))[dD]((?:[-+]?\d+)?)')
real8pattern = re.compile(r'([-+]?((?:\d+(?:\.\d*)?|\d*\.\d+))[eE]((?:[-+]?\d+)?)|(\d+\.\d*))')

_intentcallbackpattern = re.compile(r'intent\s*\(.*?\bcallback\b',re.I)
def _is_intent_callback(vdecl):
    for a in vdecl.get('attrspec',[]):
        if _intentcallbackpattern.match(a):
            return 1
    return 0

def _resolvenameargspattern(line):
    line = markouterparen(line)
    m1=nameargspattern.match(line)
    if m1: return m1.group('name'),m1.group('args'),m1.group('result')
    m1=callnameargspattern.match(line)
    if m1: return m1.group('name'),m1.group('args'),None
    return None,[],None

def analyzeline(m,case,line):
    global groupcounter,groupname,groupcache,grouplist,filepositiontext,\
           currentfilename,f77modulename,neededinterface,neededmodule,expectbegin,\
           gotnextfile,previous_context
    block=m.group('this')
    if case != 'multiline':
        previous_context = None
    if expectbegin and case not in ['begin','call','callfun','type'] \
       and not skipemptyends and groupcounter<1:
        newname=os.path.basename(currentfilename).split('.')[0]
        outmess('analyzeline: no group yet. Creating program group with name "%s".\n'%newname)
        gotnextfile=0
        groupcounter=groupcounter+1
        groupname[groupcounter]='program'
        groupcache[groupcounter]={}
        grouplist[groupcounter]=[]
        groupcache[groupcounter]['body']=[]
        groupcache[groupcounter]['vars']={}
        groupcache[groupcounter]['block']='program'
        groupcache[groupcounter]['name']=newname
        groupcache[groupcounter]['from']='fromsky'
        expectbegin=0
    if case in ['begin','call','callfun']:
        # Crack line => block,name,args,result
        block = block.lower()
        if re.match(r'block\s*data',block,re.I): block='block data'
        if re.match(r'python\s*module',block,re.I): block='python module'
        name,args,result = _resolvenameargspattern(m.group('after'))
        if name is None:
            if block=='block data':
                name = '_BLOCK_DATA_'
            else:
                name = ''
            if block not in ['interface','block data']:
                outmess('analyzeline: No name/args pattern found for line.\n')

        previous_context = (block,name,groupcounter)
        if args: args=rmbadname([x.strip() for x in markoutercomma(args).split('@,@')])
        else: args=[]
        if '' in args:
            while '' in args:
                args.remove('')
            outmess('analyzeline: argument list is malformed (missing argument).\n')

        # end of crack line => block,name,args,result
        needmodule=0
        needinterface=0

        if case in ['call','callfun']:
            needinterface=1
            if 'args' not in groupcache[groupcounter]:
                return
            if name not in groupcache[groupcounter]['args']:
                return
            for it in grouplist[groupcounter]:
                if it['name']==name:
                    return
            if name in groupcache[groupcounter]['interfaced']:
                return
            block={'call':'subroutine','callfun':'function'}[case]
        if f77modulename and neededmodule==-1 and groupcounter<=1:
            neededmodule=groupcounter+2
            needmodule=1
            needinterface=1
        # Create new block(s)
        groupcounter=groupcounter+1
        groupcache[groupcounter]={}
        grouplist[groupcounter]=[]
        if needmodule:
            if verbose>1:
                outmess('analyzeline: Creating module block %s\n'%`f77modulename`,0)
            groupname[groupcounter]='module'
            groupcache[groupcounter]['block']='python module'
            groupcache[groupcounter]['name']=f77modulename
            groupcache[groupcounter]['from']=''
            groupcache[groupcounter]['body']=[]
            groupcache[groupcounter]['externals']=[]
            groupcache[groupcounter]['interfaced']=[]
            groupcache[groupcounter]['vars']={}
            groupcounter=groupcounter+1
            groupcache[groupcounter]={}
            grouplist[groupcounter]=[]
        if needinterface:
            if verbose>1:
                outmess('analyzeline: Creating additional interface block (groupcounter=%s).\n' % (groupcounter),0)
            groupname[groupcounter]='interface'
            groupcache[groupcounter]['block']='interface'
            groupcache[groupcounter]['name']='unknown_interface'
            groupcache[groupcounter]['from']='%s:%s'%(groupcache[groupcounter-1]['from'],groupcache[groupcounter-1]['name'])
            groupcache[groupcounter]['body']=[]
            groupcache[groupcounter]['externals']=[]
            groupcache[groupcounter]['interfaced']=[]
            groupcache[groupcounter]['vars']={}
            groupcounter=groupcounter+1
            groupcache[groupcounter]={}
            grouplist[groupcounter]=[]
        groupname[groupcounter]=block
        groupcache[groupcounter]['block']=block
        if not name: name='unknown_'+block
        groupcache[groupcounter]['prefix']=m.group('before')
        groupcache[groupcounter]['name']=rmbadname1(name)
        groupcache[groupcounter]['result']=result
        if groupcounter==1:
            groupcache[groupcounter]['from']=currentfilename
        else:
            if f77modulename and groupcounter==3:
                groupcache[groupcounter]['from']='%s:%s'%(groupcache[groupcounter-1]['from'],currentfilename)
            else:
                groupcache[groupcounter]['from']='%s:%s'%(groupcache[groupcounter-1]['from'],groupcache[groupcounter-1]['name'])
        for k in groupcache[groupcounter].keys():
            if not groupcache[groupcounter][k]: del groupcache[groupcounter][k]
        groupcache[groupcounter]['args']=args
        groupcache[groupcounter]['body']=[]
        groupcache[groupcounter]['externals']=[]
        groupcache[groupcounter]['interfaced']=[]
        groupcache[groupcounter]['vars']={}
        groupcache[groupcounter]['entry']={}
        # end of creation
        if block=='type':
            groupcache[groupcounter]['varnames'] = []

        if case in ['call','callfun']: # set parents variables
            if name not in groupcache[groupcounter-2]['externals']:
                groupcache[groupcounter-2]['externals'].append(name)
            groupcache[groupcounter]['vars']=copy.deepcopy(groupcache[groupcounter-2]['vars'])
            #try: del groupcache[groupcounter]['vars'][groupcache[groupcounter-2]['name']]
            #except: pass
            try: del groupcache[groupcounter]['vars'][name][groupcache[groupcounter]['vars'][name]['attrspec'].index('external')]
            except: pass
        if block in ['function','subroutine']: # set global attributes
            try: groupcache[groupcounter]['vars'][name]=appenddecl(groupcache[groupcounter]['vars'][name],groupcache[groupcounter-2]['vars'][''])
            except: pass
            if case=='callfun': # return type
                if result and result in groupcache[groupcounter]['vars']:
                    if not name==result:
                        groupcache[groupcounter]['vars'][name]=appenddecl(groupcache[groupcounter]['vars'][name],groupcache[groupcounter]['vars'][result])
            #if groupcounter>1: # name is interfaced
            try: groupcache[groupcounter-2]['interfaced'].append(name)
            except: pass
        if block=='function':
            t=typespattern[0].match(m.group('before')+' '+name)
            if t:
                typespec,selector,attr,edecl=cracktypespec0(t.group('this'),t.group('after'))
                updatevars(typespec,selector,attr,edecl)
        if case in ['call','callfun']:
            grouplist[groupcounter-1].append(groupcache[groupcounter])
            grouplist[groupcounter-1][-1]['body']=grouplist[groupcounter]
            del grouplist[groupcounter]
            groupcounter=groupcounter-1 # end routine
            grouplist[groupcounter-1].append(groupcache[groupcounter])
            grouplist[groupcounter-1][-1]['body']=grouplist[groupcounter]
            del grouplist[groupcounter]
            groupcounter=groupcounter-1 # end interface
    elif case=='entry':
        name,args,result=_resolvenameargspattern(m.group('after'))
        if name is not None:
            if args:
                args=rmbadname([x.strip() for x in markoutercomma(args).split('@,@')])
            else: args=[]
            assert result is None,`result`
            groupcache[groupcounter]['entry'][name] = args
            previous_context = ('entry',name,groupcounter)
    elif case=='type':
        typespec,selector,attr,edecl=cracktypespec0(block,m.group('after'))
        last_name = updatevars(typespec,selector,attr,edecl)
        if last_name is not None:
            previous_context = ('variable',last_name,groupcounter)
    elif case in ['dimension','intent','optional','required','external','public','private','intrisic']:
        edecl=groupcache[groupcounter]['vars']
        ll=m.group('after').strip()
        i=ll.find('::')
        if i<0 and case=='intent':
            i=markouterparen(ll).find('@)@')-2
            ll=ll[:i+1]+'::'+ll[i+1:]
            i=ll.find('::')
            if ll[i:]=='::' and 'args' in groupcache[groupcounter]:
                outmess('All arguments will have attribute %s%s\n'%(m.group('this'),ll[:i]))
                ll = ll + ','.join(groupcache[groupcounter]['args'])
        if i<0:i=0;pl=''
        else: pl=ll[:i].strip();ll=ll[i+2:]
        ch = markoutercomma(pl).split('@,@')
        if len(ch)>1:
            pl = ch[0]
            outmess('analyzeline: cannot handle multiple attributes without type specification. Ignoring %r.\n' % (','.join(ch[1:])))
        last_name = None

        for e in [x.strip() for x in markoutercomma(ll).split('@,@')]:
            m1=namepattern.match(e)
            if not m1:
                if case in ['public','private']: k=''
                else:
                    print m.groupdict()
                    outmess('analyzeline: no name pattern found in %s statement for %s. Skipping.\n'%(case,`e`))
                    continue
            else:
                k=rmbadname1(m1.group('name'))
            if k not in edecl:
                edecl[k]={}
            if case=='dimension':
                ap=case+m1.group('after')
            if case=='intent':
                ap=m.group('this')+pl
                if _intentcallbackpattern.match(ap):
                    if k not in groupcache[groupcounter]['args']:
                        if groupcounter>1:
                            outmess('analyzeline: appending intent(callback) %s'\
                                    ' to %s arguments\n' % (k,groupcache[groupcounter]['name']))
                            if '__user__' not in groupcache[groupcounter-2]['name']:
                                outmess('analyzeline: missing __user__ module (could be nothing)\n')
                            groupcache[groupcounter]['args'].append(k)
                        else:
                            errmess('analyzeline: intent(callback) %s is ignored' % (k))
                    else:
                        errmess('analyzeline: intent(callback) %s is already'\
                                ' in argument list' % (k))
            if case in ['optional','required','public','external','private','intrisic']:
                ap=case
            if 'attrspec' in edecl[k]:
                edecl[k]['attrspec'].append(ap)
            else:
                edecl[k]['attrspec']=[ap]
            if case=='external':
                if groupcache[groupcounter]['block']=='program':
                    outmess('analyzeline: ignoring program arguments\n')
                    continue
                if k not in groupcache[groupcounter]['args']:
                    #outmess('analyzeline: ignoring external %s (not in arguments list)\n'%(`k`))
                    continue
                if 'externals' not in groupcache[groupcounter]:
                    groupcache[groupcounter]['externals']=[]
                groupcache[groupcounter]['externals'].append(k)
            last_name = k
        groupcache[groupcounter]['vars']=edecl
        if last_name is not None:
            previous_context = ('variable',last_name,groupcounter)
    elif case=='parameter':
        edecl=groupcache[groupcounter]['vars']
        ll=m.group('after').strip()[1:-1]
        last_name = None
        for e in markoutercomma(ll).split('@,@'):
            try:
                k,initexpr=[x.strip() for x in e.split('=')]
            except:
                outmess('analyzeline: could not extract name,expr in parameter statement "%s" of "%s"\n'%(e,ll));continue
            params = get_parameters(edecl)
            k=rmbadname1(k)
            if k not in edecl:
                edecl[k]={}
            if '=' in edecl[k] and (not edecl[k]['=']==initexpr):
                outmess('analyzeline: Overwriting the value of parameter "%s" ("%s") with "%s".\n'%(k,edecl[k]['='],initexpr))
            t = determineexprtype(initexpr,params)
            if t:
                if t.get('typespec')=='real':
                    tt = list(initexpr)
                    for m in real16pattern.finditer(initexpr):
                        tt[m.start():m.end()] = list(\
                            initexpr[m.start():m.end()].lower().replace('d', 'e'))
                    initexpr = ''.join(tt)
                elif t.get('typespec')=='complex':
                    initexpr = initexpr[1:].lower().replace('d','e').\
                               replace(',','+1j*(')
            try:
                v = eval(initexpr,{},params)
            except (SyntaxError,NameError,TypeError),msg:
                errmess('analyzeline: Failed to evaluate %r. Ignoring: %s\n'\
                        % (initexpr, msg))
                continue
            edecl[k]['='] = repr(v)
            if 'attrspec' in edecl[k]:
                edecl[k]['attrspec'].append('parameter')
            else: edecl[k]['attrspec']=['parameter']
            last_name = k
        groupcache[groupcounter]['vars']=edecl
        if last_name is not None:
            previous_context = ('variable',last_name,groupcounter)
    elif case=='implicit':
        if m.group('after').strip().lower()=='none':
            groupcache[groupcounter]['implicit']=None
        elif m.group('after'):
            if 'implicit' in groupcache[groupcounter]:
                impl=groupcache[groupcounter]['implicit']
            else: impl={}
            if impl is None:
                outmess('analyzeline: Overwriting earlier "implicit none" statement.\n')
                impl={}
            for e in markoutercomma(m.group('after')).split('@,@'):
                decl={}
                m1=re.match(r'\s*(?P<this>.*?)\s*(\(\s*(?P<after>[a-z-, ]+)\s*\)\s*|)\Z',e,re.I)
                if not m1:
                    outmess('analyzeline: could not extract info of implicit statement part "%s"\n'%(e));continue
                m2=typespattern4implicit.match(m1.group('this'))
                if not m2:
                    outmess('analyzeline: could not extract types pattern of implicit statement part "%s"\n'%(e));continue
                typespec,selector,attr,edecl=cracktypespec0(m2.group('this'),m2.group('after'))
                kindselect,charselect,typename=cracktypespec(typespec,selector)
                decl['typespec']=typespec
                decl['kindselector']=kindselect
                decl['charselector']=charselect
                decl['typename']=typename
                for k in decl.keys():
                    if not decl[k]: del decl[k]
                for r in markoutercomma(m1.group('after')).split('@,@'):
                    if '-' in r:
                        try: begc,endc=[x.strip() for x in r.split('-')]
                        except:
                            outmess('analyzeline: expected "<char>-<char>" instead of "%s" in range list of implicit statement\n'%r);continue
                    else: begc=endc=r.strip()
                    if not len(begc)==len(endc)==1:
                        outmess('analyzeline: expected "<char>-<char>" instead of "%s" in range list of implicit statement (2)\n'%r);continue
                    for o in range(ord(begc),ord(endc)+1):
                        impl[chr(o)]=decl
            groupcache[groupcounter]['implicit']=impl
    elif case=='data':
        ll=[]
        dl='';il='';f=0;fc=1;inp=0
        for c in m.group('after'):
            if not inp:
                if c=="'": fc=not fc
                if c=='/' and fc: f=f+1;continue
            if c=='(': inp = inp + 1
            elif c==')': inp = inp - 1
            if f==0: dl=dl+c
            elif f==1: il=il+c
            elif f==2:
                dl = dl.strip()
                if dl.startswith(','):
                    dl = dl[1:].strip()
                ll.append([dl,il])
                dl=c;il='';f=0
        if f==2:
            dl = dl.strip()
            if dl.startswith(','):
                dl = dl[1:].strip()
            ll.append([dl,il])
        vars={}
        if 'vars' in groupcache[groupcounter]:
            vars=groupcache[groupcounter]['vars']
        last_name = None
        for l in ll:
            l=[x.strip() for x in l]
            if l[0][0]==',':l[0]=l[0][1:]
            if l[0][0]=='(':
                outmess('analyzeline: implied-DO list "%s" is not supported. Skipping.\n'%l[0])
                continue
            #if '(' in l[0]:
            #    #outmess('analyzeline: ignoring this data statement.\n')
            #    continue
            i=0;j=0;llen=len(l[1])
            for v in rmbadname([x.strip() for x in markoutercomma(l[0]).split('@,@')]):
                if v[0]=='(':
                    outmess('analyzeline: implied-DO list "%s" is not supported. Skipping.\n'%v)
                    # XXX: subsequent init expressions may get wrong values.
                    #      Ignoring since data statements are irrelevant for wrapping.
                    continue
                fc=0
                while (i<llen) and (fc or not l[1][i]==','):
                    if l[1][i]=="'": fc=not fc
                    i=i+1
                i=i+1
                #v,l[1][j:i-1]=name,initvalue
                if v not in vars:
                    vars[v]={}
                if '=' in vars[v] and not vars[v]['=']==l[1][j:i-1]:
                    outmess('analyzeline: changing init expression of "%s" ("%s") to "%s"\n'%(v,vars[v]['='],l[1][j:i-1]))
                vars[v]['=']=l[1][j:i-1]
                j=i
                last_name = v
        groupcache[groupcounter]['vars']=vars
        if last_name is not None:
            previous_context = ('variable',last_name,groupcounter)
    elif case=='common':
        line=m.group('after').strip()
        if not line[0]=='/':line='//'+line
        cl=[]
        f=0;bn='';ol=''
        for c in line:
            if c=='/':f=f+1;continue
            if f>=3:
                bn = bn.strip()
                if not bn: bn='_BLNK_'
                cl.append([bn,ol])
                f=f-2;bn='';ol=''
            if f%2: bn=bn+c
            else: ol=ol+c
        bn = bn.strip()
        if not bn: bn='_BLNK_'
        cl.append([bn,ol])
        commonkey={}
        if 'common' in groupcache[groupcounter]:
            commonkey=groupcache[groupcounter]['common']
        for c in cl:
            if c[0] in commonkey:
                outmess('analyzeline: previously defined common block encountered. Skipping.\n')
                continue
            commonkey[c[0]]=[]
            for i in [x.strip() for x in markoutercomma(c[1]).split('@,@')]:
                if i: common…