Skip to content
Snippets Groups Projects
Commit ea2754e5 authored by Lennard's avatar Lennard
Browse files

Add better documentation

parent 4e1c167e
No related branches found
No related tags found
No related merge requests found
# DMS Messungen vom Silo # DMS Messungen vom Silo
## Daten
- Die Daten der Messungen bis zum Vortag können unter [Veröffentlichungen/Releases](https://gitlab.cvh-server.de/Lennard/messdatensilo/-/releases/latest) heruntergeladen werden.
- Die Daten werden als `.mat` Datein gespeichert und sind jeweils von einer Woche.
## Funktionsweise
- Ein Cron Job führt nach jeden reboot and jeden Tag um 0:00 das `scripts/run.bash` skript aus.
- `run.bash` führt dann `python3 main.py` aus, welches die Daten für einen Tag sammelt, und läd diese dann im Anschluss auf gitlab.cvh-server.de hoch.
- Das `main.py` Programm liest immer die Daten von den Arduinos, mittelt diese über einen gewissen Zeitraum und speichert diese dann in `data/data` ab.
- `main.py` stoppt dann kurz vor Mitternacht, benennt `data/data` dann in `log.Jahr-Monat-Tag_Stunde.log` um und löscht die älteste Datei, falls zu viele Datein vorhanden sind.
- Der Zeitraum, die Anzahl zu behaltene Datein und weitere Parameter von dem Programm können in `config.yml` verändert werden.
- Zusätzlich werden noch weitere Log Datein geführt:
- In `logs/*` werden logs vom `python3 main.py` geschrieben.
- In `bash.log` werden logs von `run.bash` geschrieben.
\ No newline at end of file
DataLogger: DataLogger:
backupCount: 70 # number of datalogs to keep backupCount: 70 # number of datalogs to keep
filename: data # filename for datalogs
levels: # log level for outputting to file and to stdout respectivly levels: # log level for outputting to file and to stdout respectivly
- INFO - INFO
- WARNING - WARNING
...@@ -8,11 +7,10 @@ DataLogger: ...@@ -8,11 +7,10 @@ DataLogger:
InfoLogger: InfoLogger:
backupCount: 10 # number of logs to keep backupCount: 10 # number of logs to keep
maxBytes: 100000 # size of single log in bytes maxBytes: 100000 # size of single log in bytes
filename: log # filename for logs
levels: # log level for outputting to file and to stdout respectivly levels: # log level for outputting to file and to stdout respectivly
- INFO - INFO
- WARNING - WARNING
Data: Data:
factors: [1.855, 0, 0.923, -1] # factors for the 4 dms factors: [1.855, 0, 0.923, -1] # factors for the 4 dms
delta_time: 10 # time between logging data delta_time: 30 # time between logging data
...@@ -61,7 +61,10 @@ def convert(data) -> str: ...@@ -61,7 +61,10 @@ def convert(data) -> str:
def get_offset() -> np.ndarray: def get_offset() -> np.ndarray:
files = sorted(glob.glob(str(Path.joinpath(Path(__file__).parent, "data", "data.*.log")))) """
Try and read the last logged value from the previous datalog file and use that as an offset.
"""
files = sorted(glob.glob(str(Path.joinpath(Path(__file__).parent, "data", "log.*.log"))))
if files: if files:
for file in files[::-1]: for file in files[::-1]:
...@@ -85,6 +88,9 @@ sys.excepthook = handle_exception ...@@ -85,6 +88,9 @@ sys.excepthook = handle_exception
def setup_loggers(config: Any) -> None: def setup_loggers(config: Any) -> None:
"""
Configure the two loggers. DataLogger for logging the data and InfoLogger for logging various information.
"""
global data_logger, logger, fh global data_logger, logger, fh
data_logger = logging.getLogger("data_logger") data_logger = logging.getLogger("data_logger")
data_logger.setLevel(logging.DEBUG) data_logger.setLevel(logging.DEBUG)
...@@ -92,7 +98,7 @@ def setup_loggers(config: Any) -> None: ...@@ -92,7 +98,7 @@ def setup_loggers(config: Any) -> None:
fh.append( fh.append(
TimedRotatingFileHandlerWithHeader( TimedRotatingFileHandlerWithHeader(
header=f"Timestamp,{','.join([f'dms{i+1}' for i in range(4)])},{','.join([f'temp{i+1}' for i in range(4)])},n", header=f"Timestamp,{','.join([f'dms{i+1}' for i in range(4)])},{','.join([f'temp{i+1}' for i in range(4)])},n",
filename=f"{Path(__file__).parent}/data/{config['DataLogger']['filename']}", filename=f"{Path(__file__).parent}/data/data",
when="h", when="h",
interval=23, interval=23,
backupCount=config["DataLogger"]["backupCount"], backupCount=config["DataLogger"]["backupCount"],
...@@ -110,7 +116,7 @@ def setup_loggers(config: Any) -> None: ...@@ -110,7 +116,7 @@ def setup_loggers(config: Any) -> None:
bf = logging.Formatter("{asctime}, {levelname}, [{name}.{funcName}:{lineno}]\t{message}", datefmt=r"%Y-%m-%d %H:%M:%S", style="{") bf = logging.Formatter("{asctime}, {levelname}, [{name}.{funcName}:{lineno}]\t{message}", datefmt=r"%Y-%m-%d %H:%M:%S", style="{")
fh.append( fh.append(
logging.handlers.RotatingFileHandler( logging.handlers.RotatingFileHandler(
filename=f"{Path(__file__).parent}/logs/{config['InfoLogger']['filename']}", filename=f"{Path(__file__).parent}/logs/log",
maxBytes=config["InfoLogger"]["maxBytes"], maxBytes=config["InfoLogger"]["maxBytes"],
backupCount=config["InfoLogger"]["backupCount"], backupCount=config["InfoLogger"]["backupCount"],
) )
...@@ -189,7 +195,7 @@ def main(config: Any) -> None: ...@@ -189,7 +195,7 @@ def main(config: Any) -> None:
data = np.zeros((8,)) data = np.zeros((8,))
last_write = time.time() last_write = time.time()
fh[0].doRollover() fh[0].doRollover() # rollover the current data log file
logger.warning("Finished") logger.warning("Finished")
......
#!/bin/bash #!/bin/bash
cd `dirname "$0"`/.. cd `dirname "$0"`/..
echo -e "\n\n#####################################################################"
echo $( date '+%F %H:%M:%S' ) | tee -a bash.log echo $( date '+%F %H:%M:%S' ) | tee -a bash.log
echo "" | tee -a bash.log echo "" | tee -a bash.log
scripts/./push.bash 2>&1 | tee -a bash.log scripts/./push.bash 2>&1 | tee -a bash.log
......
...@@ -17,22 +17,8 @@ python3 measure.py ...@@ -17,22 +17,8 @@ python3 measure.py
sleep 10 sleep 10
echo -e "Starting python script:\n" echo -e "Starting python script:\n"
loop_count=1
while : ; do
python3 main.py python3 main.py
return_code=$?
if [[ "$return_code" -eq 0 ]]; then
echo -e "\nPython script ended" echo -e "\nPython script ended"
break
fi
if [[ "$loop_count" -eq 10 ]]; then
echo -e "\nPython script failed too many times, trying again tomorrow"
break
fi
echo -e "Ended with error, waiting and trying again"
loop_count=$((loop_count + 1))
sleep $((loop_count * 60))
done
git pull git pull
git add . git add .
......
#!/bin/bash #!/bin/bash
# Compile and upload the DmsMessung und Temperaturmessung sketches to the
# arduinos. First getting the serial ports that each arduino is connected
# to, then checking some things and then compiling and uploading the sketches.
cd `dirname "$0"`/../sketches cd `dirname "$0"`/../sketches
fqbn=arduino:avr:uno fqbn=arduino:avr:uno
......
0% Loading or .
You are about to add 0 people to the discussion. Proceed with caution.
Please register or to comment