Thursday, October 22

Show the complete apache config file

In the Apache config file, you can use "Include" or "IncludeOptional" to include other config files. A lot of the Linux variants take advantage of that to organize the config files. For example, the default congif file of Ubuntu is in /etc/apache2/apache.conf, and it includes enabled modules, enabled sites, and configuration files this way:

IncludeOptional mods-enabled/*.conf
IncludeOptional conf-enabled/*.conf
IncludeOptional sites-enabled/*.conf

If you really want to see all the complete config settings, there is no existing tool for that. This Stack Overflow page  answered this question pretty well: You can use apachectl -S to see the settings of Virtual Host, or apachectl -M to see the loaded modules, but to see all settings, there is no such tool, you will have to go through all the files , starting from familiar yourself with the  general structure of the httpd config files. 

So I created this python program to generate the complete apache config file:

__author__ = 'ben'
import sys, os, os.path, logging, fnmatch, re

def Help():
    print("Usage: python inputfile[default:/etc/apache2/apache2.conf] outputfile[default:/tmp/apache2.combined.conf")

def InputParameter():
    if len(sys.argv) <> 3:
        return "/etc/apache2/apache2.conf", "/tmp/apache2.combined.conf"
    return sys.argv[1], sys.argv[2]

def ProcessMultipleFiles(InputFiles):
    if InputFiles.endswith('/'):              #Updated as Pierrick's comment
        InputFiles = InputFiles + "*"
    Content = ''
    LocalFolder = os.path.dirname(InputFiles)
    basenamePattern = os.path.basename(InputFiles)
    for root, dirs, files in os.walk(LocalFolder):
        for filename in fnmatch.filter(files, basenamePattern):
            Content += ProcessInput(os.path.join(root, filename))
    return Content

def RemoveExcessiveLinebreak(s):
    Length = len(s)
    s = s.replace(os.linesep + os.linesep + os.linesep, os.linesep + os.linesep)
    NewLength = len(s)
    if NewLength < Length:
        s = RemoveExcessiveLinebreak(s)
    return s

def ProcessInput(InputFile):
    global ServerRoot

    Content = ''
    if logging.root.isEnabledFor(logging.DEBUG):
        Content = '# Start of ' + InputFile + os.linesep
    with open(InputFile, 'r') as infile:
        for line in infile:
            stripline = line.strip(' \t')
            if stripline.startswith('#'):
            searchroot ='ServerRoot\s+(\S+)', stripline, re.I)      #search for ServerRoot
            if searchroot:
                ServerRoot ='"')
      "ServerRoot: " + ServerRoot)
            if stripline.lower().startswith('include'):
                match = stripline.split()
                if len(match) == 2:
                    IncludeFiles = match[1]
                    IncludeFiles = IncludeFiles.strip('"') #Inserted according to V's comment.
                    if not IncludeFiles.startswith('/'):
                        IncludeFiles = os.path.join(ServerRoot, IncludeFiles)

                    Content += ProcessMultipleFiles(IncludeFiles) + os.linesep
                    Content += line     # if it is not pattern of 'include(optional) path', then continue.
                Content += line
    Content = RemoveExcessiveLinebreak(Content)
    if logging.root.isEnabledFor(logging.DEBUG):
        Content += '# End of ' + InputFile + os.linesep + os.linesep
    return Content

if __name__ == "__main__":
    logging.basicConfig(level=logging.DEBUG, format='[%(asctime)s][%(levelname)s]:%(message)s')
    InputFile, OutputFile = InputParameter()
        ServerRoot = os.path.dirname(InputFile)
        Content = ProcessInput(InputFile)
    except Exception as e:
        logging.error("Failed to process " + InputFile,  exc_info=True)

        with open(OutputFile, 'w') as outfile:
    except Exception as e:
        logging.error("Failed to write to " + outfile,  exc_info=True)
        exit(1)"Done writing " + OutputFile)

The usage is simple: Run it as "python ". Since there is no additional parameters given, it will retrieve the default Ubuntu apache config file from  /etc/apache2/apache2.conf and generate the result complete config file in /tmp/apache2.combined.conf. If your config file is in different location, then give the input file and output file location. For example, RHEL has the config file in /etc/httpd/conf/httpd.conf, then you can run "python /etc/httpd/conf/httpd.conf /tmp/apache2.combined.conf ".

Note: Apache server-info page also provide similar information, but not in the config file format. It is in human readable format. The page works only when it is open from the same computer.



At April 21, 2016 12:23 PM, Blogger Unknown said...

Nice, but is there a way to dump the configs if macro_module is being used.


At May 05, 2016 12:50 PM, Blogger Unknown said...

It's a nice looking idea, but doesn't seem to be including any included files... is this working for other people?

At May 05, 2016 1:46 PM, Blogger Unknown said...

Let me correct/clarify that a bit... It seems to work if the include is a full path and not a relative path to the httpd config root (which works for apache and is allowed as far as I know).

This works
Include /etc/httpd/conf/vhosts.d/*.conf

This does not work (although apache is OK with it)
Include vhosts.d/*.conf

For some reason, this does not work (from a whm/cpanel httpd.conf)
Include "/etc/apache2/conf.d/userdata/std/2_4/username/*.conf"

I'm guessing the issue in the cpanel example might be the quotations around the string

At September 29, 2016 6:23 PM, Blogger Ben, blogging said...

Hi V,

Confirmed that path with quotations will not work. But the relative path definitely works well for me.

I will update the program to deal with quotations later. With source code, you should be able to do that as well :)

At October 03, 2016 10:05 AM, Blogger Ben, blogging said...

Inserted one line "IncludeFiles = IncludeFiles.strip('"')" to deal with the config file with quotation marks according to V's comment.

Thanks, V.

At January 27, 2017 3:57 AM, Blogger Unknown said...

It does not work with folder includes. Like "Include conf.d/"
As workaround, I added the next lines in ProcessMultipleFiles:

if InputFiles.endswith('/'):
InputFiles = InputFiles + "*"

Also, the script may need a supplementary optional argument to specify relative path root. The script does not work on RHEL based systems. There, the base config path is /etc/httpd/conf/httpd.conf and root is /etc/httpd/.


At January 27, 2017 12:27 PM, Blogger Ben, blogging said...

Thank you, Pierrick. Confirmed that my version did not work with "Include conf.d/" scenario and your update was good, so I updated the post accordingly.

About the RHEL and optional relative path root, I will need that environment to try it out. Thanks.

At January 27, 2017 1:23 PM, Blogger Ben, blogging said...

Pierrick, after looking into an RHEL server, I found the reason was that my code ignored "ServerRoot" that Apache used to identify the root of config file ("relative path root" of your comment).

By adding the logic of ServerRoot (making a global variable, make it default as the folder of httpd.conf but available to be modified as reading the config file), the code is working well in both RHEL environment which has ServerRoot specified, and Debian environment.

Thanks for pointing out the issue!

At January 27, 2017 1:26 PM, Blogger Ben, blogging said...

After revisiting the previous comment, I think maybe Verdon's frustration come from the same source: the ServerRoot in RHEL environment was not handled properly so the relative path was not correctly processed.

Please use the new version!


<< Home