Skip to content
Snippets Groups Projects
Commit 2da5c602 authored by Raspberry's avatar Raspberry
Browse files

Cleanup

parent 7e8bd7cc
No related branches found
No related tags found
No related merge requests found
Pipeline #776 passed
The MIT License (MIT)
Copyright (c) 2019 Marvin Pinto and contributors
Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:
The above copyright notice and this permission notice shall be included in
all copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
THE SOFTWARE.
# GitHub Automatic Releases
This action simplifies the GitHub release process by automatically uploading assets, generating changelogs, handling pre-releases, and so on.
## Contents
1. [Usage Examples](#usage-examples)
1. [Supported Parameters](#supported-parameters)
1. [Event Triggers](#event-triggers)
1. [Versioning](#versioning)
1. [How to get help](#how-to-get-help)
1. [License](#license)
> **NOTE**: The `marvinpinto/action-automatic-releases` repository is an automatically generated mirror of the [marvinpinto/actions](https://github.com/marvinpinto/actions) monorepo containing this and other actions. Please file issues and pull requests over there.
## Usage Examples
### Automatically generate a pre-release when changes land on master
This example workflow will kick in as soon as changes land on `master`. After running the steps to build and test your project:
1. It will create (or replace) a git tag called `latest`.
1. Generate a changelog from all the commits between this, and the previous `latest` tag.
1. Generate a new release associated with the `latest` tag (removing any previous associated releases).
1. Update this new release with the specified title (e.g. `Development Build`).
1. Upload `LICENSE.txt` and any `jar` files as release assets.
1. Mark this release as a `pre-release`.
You can see a working example of this workflow over at [marvinpinto/actions](https://github.com/marvinpinto/actions/releases/tag/latest).
```yaml
---
name: "pre-release"
on:
push:
branches:
- "master"
jobs:
pre-release:
name: "Pre Release"
runs-on: "ubuntu-latest"
steps:
# ...
- name: "Build & test"
run: |
echo "done!"
- uses: "marvinpinto/action-automatic-releases@latest"
with:
repo_token: "${{ secrets.GITHUB_TOKEN }}"
automatic_release_tag: "latest"
prerelease: true
title: "Development Build"
files: |
LICENSE.txt
*.jar
```
### Create a new GitHub release when tags are pushed to the repository
Similar to the previous example, this workflow will kick in as soon as new tags are pushed to GitHub. After building & testing your project:
1. Generate a changelog from all the commits between this and the previous [semver-looking](https://semver.org/) tag.
1. Generate a new release and associate it with this tag.
1. Upload `LICENSE.txt` and any `jar` files as release assets.
Once again there's an example of this over at [marvinpinto/actions](https://github.com/marvinpinto/actions/releases/latest).
```yaml
---
name: "tagged-release"
on:
push:
tags:
- "v*"
jobs:
tagged-release:
name: "Tagged Release"
runs-on: "ubuntu-latest"
steps:
# ...
- name: "Build & test"
run: |
echo "done!"
- uses: "marvinpinto/action-automatic-releases@latest"
with:
repo_token: "${{ secrets.GITHUB_TOKEN }}"
prerelease: false
files: |
LICENSE.txt
*.jar
```
## Supported Parameters
| Parameter | Description | Default |
| ----------------------- | ----------------------------------------------------------------------------- | -------- |
| `repo_token`\*\* | GitHub Action token, e.g. `"${{ secrets.GITHUB_TOKEN }}"`. | `null` |
| `draft` | Mark this release as a draft? | `false` |
| `prerelease` | Mark this release as a pre-release? | `true` |
| `automatic_release_tag` | Tag name to use for automatic releases, e.g `latest`. | `null` |
| `title` | Release title; defaults to the tag name if none specified. | Tag Name |
| `files` | Files to upload as part of the release assets. | `null` |
| `changelog_file` | Text you want to put in the release/changelog info. (Disables auto changelog) | `null` |
## Outputs
The following output values can be accessed via `${{ steps.<step-id>.outputs.<output-name> }}`:
| Name | Description | Type |
| ------------------------ | ------------------------------------------------------ | ------ |
| `automatic_releases_tag` | The release tag this action just processed | string |
| `upload_url` | The URL for uploading additional assets to the release | string |
### Notes:
- Parameters denoted with `**` are required.
- The `files` parameter supports multi-line [glob](https://github.com/isaacs/node-glob) patterns, see repository examples.
## Event Triggers
The GitHub Actions framework allows you to trigger this (and other) actions on _many combinations_ of events. For example, you could create specific pre-releases for release candidate tags (e.g `*-rc*`), generate releases as changes land on master (example above), nightly releases, and much more. Read through [Workflow syntax for GitHub Actions](https://help.github.com/en/articles/workflow-syntax-for-github-actions) for ideas and advanced examples.
## Versioning
Every commit that lands on master for this project triggers an automatic build as well as a tagged release called `latest`. If you don't wish to live on the bleeding edge you may use a stable release instead. See [releases](../../releases/latest) for the available versions.
```yaml
- uses: "marvinpinto/action-automatic-releases@<VERSION>"
```
## How to get help
The main [README](https://github.com/marvinpinto/actions/blob/master/README.md) for this project has a bunch of information related to debugging & submitting issues. If you're still stuck, try and get a hold of me on [keybase](https://keybase.io/marvinpinto) and I will do my best to help you out.
## License
The source code for this project is released under the [MIT License](/LICENSE). This project is not associated with GitHub.
name: "Automatic Releases"
author: "marvinpinto"
description: "Automate the GitHub release process with assets, changelogs, pre-releases, and more"
inputs:
repo_token:
description: "GitHub secret token"
required: true
automatic_release_tag:
description: "Git tag (for automatic releases)"
required: false
draft:
description: "Should this release be marked as a draft?"
required: false
default: false
prerelease:
description: "Should this release be marked as a pre-release?"
required: false
default: true
title:
description: "Release title (for automatic releases)"
required: false
files:
description: "Assets to upload to the release"
required: false
changelog_file:
description: "Text you want to put in the release/changelog info. (Disables auto changelog)"
required: false
outputs:
automatic_releases_tag:
description: "The release tag this action just processed"
upload_url:
description: "The URL for uploading additional assets to the release"
runs:
using: "node12"
main: "dist/index.js"
branding:
icon: "git-merge"
color: "red"
This diff is collapsed.
name: Convert data to .mat file(s)
on:
schedule:
- cron: "0 1 * * *"
jobs:
build:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v3
with:
lfs: 'true'
- name: Set up Python 3.10
uses: actions/setup-python@v3
with:
python-version: "3.10"
- name: Install dependencies
run: |
python3 -m pip install --upgrade pip
python3 -m pip install numpy scipy
- name: Convert data
run: |
python3 convert.py
- name: release
uses: "./.github/actions/automatic-releases"
with:
repo_token: ${{ secrets.GITHUB_TOKEN }}
prerelease: false
title: "Data"
automatic_release_tag: latest
changelog_file: ./CHANGELOG.md
files: |
data.zip
#
# Example of how to profile a Python app with multiple processes
# by logging events and opening the resulting trace file in chrome://tracing.
#
# pip install multiprocessing_logging
from functools import wraps
import json
import logging
import os
import time
import threading
import traceback
from typing import Any
import serial
import serial.serialutil
import sys
import datetime
import yaml
from pathlib import Path
import numpy as np
from multiprocessing_logging import install_mp_handler
from main import logger, data_logger, fh, get_offset, setup_loggers
# separate logger that only stores events into a file
prof_logger = logging.getLogger("profiling")
# not not propagate to root logger, we want to store this separately
prof_logger.propagate = False
handler = logging.FileHandler("profiling.log", "w+")
handler.setFormatter(logging.Formatter("%(message)s"))
prof_logger.addHandler(handler)
install_mp_handler(prof_logger)
def log_event(**kwargs):
prof_logger.debug(json.dumps(kwargs))
def time_usec() -> int:
return int(round(1e6 * time.time()))
base_info = {
"pid": os.getpid(),
"tid": threading.current_thread().ident,
}
def log_profile(category: str = None):
def decorator(f):
@wraps(f)
def wrapper(*args, **kwargs):
# format compatible with chrome://tracing
# info: https://www.gamasutra.com/view/news/176420/Indepth_Using_Chrometracing_to_view_your_inline_profiling_data.php
args_str = {i: f"{arg}" for i, arg in enumerate(args)}
start_time = time_usec()
log_event(ph="B", ts=start_time, name=f.__name__, cat=category, args=args_str, **base_info)
result = f(*args, **kwargs)
end_time = time_usec()
duration = end_time - start_time
# TODO: duration could possibly be computed afterwards (if we can pair the events correctly)
log_event(ph="E", ts=end_time, duration=duration, name=f.__name__, cat=category, args=args_str, **base_info)
return result
return wrapper
return decorator
def convert_log_to_trace(log_file, trace_file):
with open(trace_file, "wt") as output, open(log_file, "rt") as input:
events = [json.loads(line) for line in input]
json.dump({"traceEvents": events}, output)
@log_profile("read_value")
def read_value(connection: serial.Serial) -> bytes:
return connection.readline()
@log_profile("read")
def read(connection: serial.Serial, data: np.ndarray, off: int):
for i in range(4):
recv = read_value(connection)
data[i + off] += float(convert(recv))
@log_profile("write")
def write(connection: serial.Serial):
connection.write(1)
@log_profile("offset")
def offset(connection: serial.Serial) -> int:
return 0 if int(convert(connection.readline())) == 1.0 else 4
@log_profile("write_data")
def write_data(data: np.ndarray, n: int, factors: np.ndarray, offsets: np.ndarray):
data_logger.info(",".join([f"{(value/n) * factors[i] - offsets[i]:.5f}" for i, value in enumerate(data)]) + f",{n}")
logger.debug("Wrote data")
def convert(data) -> str:
return str(data).replace("b'", "").replace("'", "").replace("\\r\\n", "")
@log_profile("get_data")
def get_data(con1: serial.Serial, con2: serial.Serial) -> np.ndarray:
data = np.zeros((8,))
try:
for connection in [con1, con2]:
write(connection)
off = offset(connection)
read(connection, data, off)
except (TypeError, ValueError):
# may occur if no data was read over serial
logger.info(f"Didn't receive data from arduino", exc_info=True)
return data
@log_profile("loop")
def loop(con1: serial.Serial, con2: serial.Serial, factors: np.ndarray, offsets: np.ndarray):
last_write = time.time()
delta_time = 30
n = 0
data = np.zeros((8,))
while time.time() - last_write < delta_time:
data += get_data(con1, con2)
n += 1
write_data(data, n, factors, offsets)
@log_profile("main")
def main(config: Any) -> None:
print("Starting")
Path(f"{Path(__file__).parent}/test_data").mkdir(parents=True, exist_ok=True)
Path(f"{Path(__file__).parent}/test_logs").mkdir(parents=True, exist_ok=True)
setup_loggers(config, "test_data", "test_logs")
delta_time = config["Data"]["delta_time"] # log averaged out data every n seconds
end_time = datetime.datetime.combine(datetime.date.today(), datetime.time(1, 0, 0, 0))
logger.warning("Starting")
factors: np.ndarray = np.hstack((np.array(config["Data"]["factors"]), np.ones((4,))))
offsets: np.ndarray = np.hstack((get_offset(), np.zeros((4,))))
logger.info(
f"Factors: {', '.join(f'{factor:.3f}' for factor in factors[:4])}, Offset: {', '.join(f'{offset:.3f}' for offset in offsets[:4])}"
)
with serial.Serial("/dev/ttyACM0", 9600, timeout=3) as con1, serial.Serial("/dev/ttyACM1", 9600, timeout=3) as con2:
for _ in range(100):
loop(con1, con2, factors, offsets)
fh[0].doRollover() # rollover the current data log file
logger.warning("Finished")
if __name__ == "__main__":
main(yaml.safe_load(open(f"{Path(__file__).parent}/config.yml")))
convert_log_to_trace("profiling.log", "profiling_trace.json")
0% Loading or .
You are about to add 0 people to the discussion. Proceed with caution.
Please register or to comment