Confucius said: He who works hard gets well paid.
Thursday, October 22
Show the complete apache config file
In the Apache config file, you can use "Include" or "IncludeOptional" to include other config files. A lot of the Linux variants take advantage of that to organize the config files. For example, the default congif file of Ubuntu is in /etc/apache2/apache.conf, and it includes enabled modules, enabled sites, and configuration files this way:
If you really want to see all the complete config settings, there is no existing tool for that. This Stack Overflow page answered this question pretty well: You can use apachectl -S to see the settings of Virtual Host, or apachectl -M to see the loaded modules, but to see all settings, there is no such tool, you will have to go through all the files , starting from familiar yourself with the general structure of the httpd config files.
So I created this python program to generate the complete apache config file:
__author__ = 'ben'
import sys, os, os.path, logging, fnmatch
print("Usage: python CombineApacheConfig.py inputfile[default:/etc/apache2/apache2.conf] outputfile[default:/tmp/apache2.combined.conf")
if len(sys.argv) <> 3:
return "/etc/apache2/apache2.conf", "/tmp/apache2.combined.conf"
return sys.argv, sys.argv
Content = ''
LocalFolder = os.path.dirname(InputFiles)
basenamePattern = os.path.basename(InputFiles)
for root, dirs, files in os.walk(LocalFolder):
for filename in fnmatch.filter(files, basenamePattern):
Content += ProcessInput(os.path.join(root, filename))
Length = len(s)
s = s.replace(os.linesep + os.linesep + os.linesep, os.linesep + os.linesep)
NewLength = len(s)
if NewLength < Length:
s = RemoveExcessiveLinebreak(s)
Content = ''
Content = '# Start of ' + InputFile + os.linesep
with open(InputFile, 'r') as infile:
for line in infile:
stripline = line.strip(' \t')
match = stripline.split()
if len(match) == 2:
IncludeFiles = match
if not IncludeFiles.startswith('/'):
LocalFolder = os.path.dirname(InputFile)
IncludeFiles = os.path.join(LocalFolder, IncludeFiles)
Content += ProcessMultipleFiles(IncludeFiles) + os.linesep
Content += line # if it is not pattern of 'include(optional) path', then continue.
Content += line
Content = RemoveExcessiveLinebreak(Content)
Content += '# End of ' + InputFile + os.linesep + os.linesep
if __name__ == "__main__":
InputFile, OutputFile = InputParameter()
Content = ProcessInput(InputFile)
except Exception as e:
logging.error("Failed to process " + InputFile, exc_info=True)
with open(OutputFile, 'w') as outfile:
except Exception as e:
logging.error("Failed to write to " + outfile, exc_info=True)
logging.info("Done writing " + OutputFile)
The usage is simple: Run it as "python CombineApacheConfig.py ". Since there is no additional parameters given, it will retrieve the default Ubuntu apache config file from /etc/apache2/apache2.conf and generate the result complete config file in /tmp/apache2.combined.conf. If your config file is in different location, then give the input file and output file location.
Note: Apache server-info page http://127.0.0.1/server-info also provide similar information, but not in the config file format. It is in human readable format. The page works only when it is open from the same computer.
1, If the program starts at 8:20AM, the TimedRotatingFileHandler will restart a new log file at 9:20AM, but my OnMarkRotatingFileHandler will restart a new log file at 9:00AM.
2, If the previous log file was last modified at 8:58 AM, if you start the program with old handler after 9:58, it will rotate the log; if you start the program before 9:58, say 9:55 the program will just append the log entries into the existing log until 10:55, if your program runs for that long.
With my new handler, now at 9:00AM, when the program starts, the new handler will rotate the log file to generate new log file.
__author__ = 'ben'
def __init__(self, filename, when='h', interval=1, backupCount=0, encoding=None, delay=False, utc=False):
#super().__init__() # in Python 2 use super(D, self).__init__()
super(OnMarkRotatingFileHandler, self).__init__(filename, when, interval, backupCount, encoding, delay, utc)
def floor_to(self, num, scale):
return int(num/scale) * scale
def computeRollover(self, currentTime):
temp_result = super(OnMarkRotatingFileHandler, self).computeRollover(currentTime)
if not self.when.startswith('W'):
result = self.floor_to(temp_result, self.interval)
result = temp_result # need to find out the first date of time (is it 1970/1/1?), what weekday that is.
Most methods are inherited from the TimedRotatingFileHandler. The W0/W1.../W6 options are not implemented yet. But you get the ideal.
To use it, place the logHandler.py with your code, then you can either import it and load it as
h = logHandler.OnMarkRotatingFileHandler ("filename")
or you can use it in the logging.ini:
then load it in logging.config like every other handler does:
The TimedRotatingFileHandler of python logging system
The name of TimedRotatingFileHandler is like the best for my need: rotate every day, keep 30 days of copy. I know the logrotate of Linux system provides the same service, but I prefer to make the program easy to install and run without System Admin (root) involving in the process. Another reason I am not using logrotate is that when my program is writing into log at midnight, logrotate would perform "mv myprogram.log myprogram.log.20151015", and my program is still writing into that file which has a new name now: myprogram.log.20151015, while I still have another program trying to access myprogram.log for display purpose, then that program will fail.
So I want to have a native logging handler to do log rotating, and change to a new file of "myprogram.log" at midnight. The TimedRotatingFileHandler looks perfect to me.
But it is not.
The TimedRotatingFileHandler only rotate the log when the program has been running for more than 24 hours (Assuming the interval setting is 1 Day ), or if the log file has not been modified (before loading) for more than 24 hours. You can check the code, but that is what it is. If your program runs every day from 1AM to 11PM, then the log is never rotate, it keep appending forever. It is using the last modified time of the log file , or current time when the file doesn't exist, to decide when to rotate.
One limitation is that in Linux there is not "file creation time" or "birth time". Most file system don't support this file attribute. I guess ext4 does, under some configuration, but we can't rely on it.