Skip to content
Snippets Groups Projects
Commit 209a8292 authored by Austin Anderson's avatar Austin Anderson
Browse files

Reusablity refactor

Big refactor to support general reusability for other ecosystem or
non-TF projects. Also includes the TensorFlow Apache License and some
other tweaks. Notably removes the little timer next to the
last-refreshed indicator because CSS animations cause a HUGE CPU hit on
a lot of hardware.

Also added feature to totally reset or import data.
parent 090ef48a
No related branches found
No related tags found
No related merge requests found
Showing with 700 additions and 172 deletions
......@@ -20,55 +20,101 @@ on:
schedule:
- cron: '*/5 * * * *'
workflow_dispatch:
inputs:
gist:
required: false
description: (OPTIONAL) Overwrite "previous" data. Format is a hexadecimal gist ID. Gist should have files of JSON data, named same as matrix config / yaml files, extracted from e.g. merged.json. Can be empty or excluded.
reset:
required: false
description: (OPTIONAL) Reset all previous data. Type "yes reset" to enable.
jobs:
dashboard:
dashboards:
if: github.repository == 'tensorflow/build' # Don't do this in forks
name: Generate Dashboard
runs-on: ubuntu-latest
environment:
name: github-pages
url: ${{ steps.deployment.outputs.page_url }}
strategy:
matrix:
config: [ tensorflow ]
permissions:
contents: read
pages: write
id-token: write
steps:
- name: "Checkout code"
- name: Checkout code
uses: actions/checkout@755da8c3cf115ac066823e79a1e1788f8940201b # v3.2.0
with:
persist-credentials: false
- name: Download artifact
id: download-artifact
- name: Download previous workflow's data
if: "${{ github.event.inputs.reset != 'yes reset' }}"
uses: dawidd6/action-download-artifact@v2
with:
path: tf_oss_dashboard
workflow_conclusion: success
if_no_artifact_found: warn
- name: Overwrite data, if provided
if: "${{ github.event.inputs.gist != '' }}"
run: |
git clone "https://gist.github.com/${{github.event.inputs.gist}}.git" gist
cd gist
if [[ -s ${{matrix.config}} ]]; then
mv ${{matrix.config}} ../tf_oss_dashboard/${{matrix.config}}/old.json
fi
- name: Run Script
id: run-script
env:
GITHUB_TOKEN: ${{secrets.GITHUB_TOKEN}}
run: cd tf_oss_dashboard; ./script.sh
run: ./merge_and_generate.sh ${{matrix.config}}
working-directory: tf_oss_dashboard
- name: "Upload merged data"
- name: Upload ongoing data and dashboard output
uses: actions/upload-artifact@83fd05a356d7e2593de66fc9913b3002723633cb # v3.1.1
with:
name: Merged Data
path: tf_oss_dashboard/old.json
name: ${{matrix.config}}
path: tf_oss_dashboard/${{matrix.config}}
retention-days: 5
- name: Setup Pages
uses: actions/configure-pages@v3
assemble:
if: github.repository == 'tensorflow/build' # Don't do this in forks
name: Assemble Dashboards
needs: dashboards
runs-on: ubuntu-latest
environment:
name: github-pages
url: ${{ steps.deployment.outputs.page_url }}
permissions:
contents: read
pages: write
id-token: write
steps:
- name: "Checkout code"
uses: actions/checkout@755da8c3cf115ac066823e79a1e1788f8940201b # v3.2.0
with:
persist-credentials: false
# Get all artifacts into current dir
- uses: actions/download-artifact@v3
with:
path: artifacts
- name: Move dashboard around, include all generated files
run: cd tf_oss_dashboard; mkdir .site; mv -t .site *
run: |
mv artifacts/tensorflow .site
mv -t .site tf_oss_dashboard/*
mv -t .site artifacts/*
- name: Setup Pages
uses: actions/configure-pages@v3
- name: Upload pages artifact
uses: actions/upload-pages-artifact@v1
with:
path: './tf_oss_dashboard/.site'
path: '.site'
- name: Deploy to GitHub Pages
id: deployment
......
# Copyright 2023 The TensorFlow Authors. All Rights Reserved.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
# ============================================================================
name: Continuous job tester
on:
schedule:
- cron: '*/5 * * * *'
workflow_dispatch:
jobs:
dashboards:
if: github.repository == 'angerson/build' # Don't do this yet
name: A continuous job
runs-on: ubuntu-latest
permissions:
statuses: write
strategy:
matrix:
config: [ tensorflow, jax ]
steps:
- uses: myrotvorets/set-commit-status-action@master
with:
status: ${{ job.status }}
sha: ${{ github.sha }}
Apache License
Version 2.0, January 2004
http://www.apache.org/licenses/
TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION
1. Definitions.
"License" shall mean the terms and conditions for use, reproduction,
and distribution as defined by Sections 1 through 9 of this document.
"Licensor" shall mean the copyright owner or entity authorized by
the copyright owner that is granting the License.
"Legal Entity" shall mean the union of the acting entity and all
other entities that control, are controlled by, or are under common
control with that entity. For the purposes of this definition,
"control" means (i) the power, direct or indirect, to cause the
direction or management of such entity, whether by contract or
otherwise, or (ii) ownership of fifty percent (50%) or more of the
outstanding shares, or (iii) beneficial ownership of such entity.
"You" (or "Your") shall mean an individual or Legal Entity
exercising permissions granted by this License.
"Source" form shall mean the preferred form for making modifications,
including but not limited to software source code, documentation
source, and configuration files.
"Object" form shall mean any form resulting from mechanical
transformation or translation of a Source form, including but
not limited to compiled object code, generated documentation,
and conversions to other media types.
"Work" shall mean the work of authorship, whether in Source or
Object form, made available under the License, as indicated by a
copyright notice that is included in or attached to the work
(an example is provided in the Appendix below).
"Derivative Works" shall mean any work, whether in Source or Object
form, that is based on (or derived from) the Work and for which the
editorial revisions, annotations, elaborations, or other modifications
represent, as a whole, an original work of authorship. For the purposes
of this License, Derivative Works shall not include works that remain
separable from, or merely link (or bind by name) to the interfaces of,
the Work and Derivative Works thereof.
"Contribution" shall mean any work of authorship, including
the original version of the Work and any modifications or additions
to that Work or Derivative Works thereof, that is intentionally
submitted to Licensor for inclusion in the Work by the copyright owner
or by an individual or Legal Entity authorized to submit on behalf of
the copyright owner. For the purposes of this definition, "submitted"
means any form of electronic, verbal, or written communication sent
to the Licensor or its representatives, including but not limited to
communication on electronic mailing lists, source code control systems,
and issue tracking systems that are managed by, or on behalf of, the
Licensor for the purpose of discussing and improving the Work, but
excluding communication that is conspicuously marked or otherwise
designated in writing by the copyright owner as "Not a Contribution."
"Contributor" shall mean Licensor and any individual or Legal Entity
on behalf of whom a Contribution has been received by Licensor and
subsequently incorporated within the Work.
2. Grant of Copyright License. Subject to the terms and conditions of
this License, each Contributor hereby grants to You a perpetual,
worldwide, non-exclusive, no-charge, royalty-free, irrevocable
copyright license to reproduce, prepare Derivative Works of,
publicly display, publicly perform, sublicense, and distribute the
Work and such Derivative Works in Source or Object form.
3. Grant of Patent License. Subject to the terms and conditions of
this License, each Contributor hereby grants to You a perpetual,
worldwide, non-exclusive, no-charge, royalty-free, irrevocable
(except as stated in this section) patent license to make, have made,
use, offer to sell, sell, import, and otherwise transfer the Work,
where such license applies only to those patent claims licensable
by such Contributor that are necessarily infringed by their
Contribution(s) alone or by combination of their Contribution(s)
with the Work to which such Contribution(s) was submitted. If You
institute patent litigation against any entity (including a
cross-claim or counterclaim in a lawsuit) alleging that the Work
or a Contribution incorporated within the Work constitutes direct
or contributory patent infringement, then any patent licenses
granted to You under this License for that Work shall terminate
as of the date such litigation is filed.
4. Redistribution. You may reproduce and distribute copies of the
Work or Derivative Works thereof in any medium, with or without
modifications, and in Source or Object form, provided that You
meet the following conditions:
(a) You must give any other recipients of the Work or
Derivative Works a copy of this License; and
(b) You must cause any modified files to carry prominent notices
stating that You changed the files; and
(c) You must retain, in the Source form of any Derivative Works
that You distribute, all copyright, patent, trademark, and
attribution notices from the Source form of the Work,
excluding those notices that do not pertain to any part of
the Derivative Works; and
(d) If the Work includes a "NOTICE" text file as part of its
distribution, then any Derivative Works that You distribute must
include a readable copy of the attribution notices contained
within such NOTICE file, excluding those notices that do not
pertain to any part of the Derivative Works, in at least one
of the following places: within a NOTICE text file distributed
as part of the Derivative Works; within the Source form or
documentation, if provided along with the Derivative Works; or,
within a display generated by the Derivative Works, if and
wherever such third-party notices normally appear. The contents
of the NOTICE file are for informational purposes only and
do not modify the License. You may add Your own attribution
notices within Derivative Works that You distribute, alongside
or as an addendum to the NOTICE text from the Work, provided
that such additional attribution notices cannot be construed
as modifying the License.
You may add Your own copyright statement to Your modifications and
may provide additional or different license terms and conditions
for use, reproduction, or distribution of Your modifications, or
for any such Derivative Works as a whole, provided Your use,
reproduction, and distribution of the Work otherwise complies with
the conditions stated in this License.
5. Submission of Contributions. Unless You explicitly state otherwise,
any Contribution intentionally submitted for inclusion in the Work
by You to the Licensor shall be under the terms and conditions of
this License, without any additional terms or conditions.
Notwithstanding the above, nothing herein shall supersede or modify
the terms of any separate license agreement you may have executed
with Licensor regarding such Contributions.
6. Trademarks. This License does not grant permission to use the trade
names, trademarks, service marks, or product names of the Licensor,
except as required for reasonable and customary use in describing the
origin of the Work and reproducing the content of the NOTICE file.
7. Disclaimer of Warranty. Unless required by applicable law or
agreed to in writing, Licensor provides the Work (and each
Contributor provides its Contributions) on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
implied, including, without limitation, any warranties or conditions
of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A
PARTICULAR PURPOSE. You are solely responsible for determining the
appropriateness of using or redistributing the Work and assume any
risks associated with Your exercise of permissions under this License.
8. Limitation of Liability. In no event and under no legal theory,
whether in tort (including negligence), contract, or otherwise,
unless required by applicable law (such as deliberate and grossly
negligent acts) or agreed to in writing, shall any Contributor be
liable to You for damages, including any direct, indirect, special,
incidental, or consequential damages of any character arising as a
result of this License or out of the use or inability to use the
Work (including but not limited to damages for loss of goodwill,
work stoppage, computer failure or malfunction, or any and all
other commercial damages or losses), even if such Contributor
has been advised of the possibility of such damages.
9. Accepting Warranty or Additional Liability. While redistributing
the Work or Derivative Works thereof, You may choose to offer,
and charge a fee for, acceptance of support, warranty, indemnity,
or other liability obligations and/or rights consistent with this
License. However, in accepting such obligations, You may act only
on Your own behalf and on Your sole responsibility, not on behalf
of any other Contributor, and only if You agree to indemnify,
defend, and hold each Contributor harmless for any liability
incurred by, or claims asserted against, such Contributor by reason
of your accepting any such warranty or additional liability.
END OF TERMS AND CONDITIONS
APPENDIX: How to apply the Apache License to your work.
To apply the Apache License to your work, attach the following
boilerplate notice, with the fields enclosed by brackets "[]"
replaced with your own identifying information. (Don't include
the brackets!) The text should be enclosed in the appropriate
comment syntax for the file format. We also recommend that a
file or class name and description of purpose be included on the
same "printed page" as the copyright notice for easier
identification within third-party archives.
Copyright [yyyy] [name of copyright owner]
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
......@@ -6,9 +6,78 @@ Maintainer: @angerson (TensorFlow, SIG Build)
* * *
This is an experimental dashboard that scrapes the GitHub GraphQL API to
display all statuses reported on TensorFlow commits. It is deployed to
This is a dashboard that scrapes the GitHub GraphQL API to
display all statuses on a repository's commits. It requires no backend, no
maintenance and works great on GitHub Pages. It is deployed to
https://tensorflow.github.io/build/ via https://github.com/tensorflow/build/tree/master/.github/workflows/dashboard.yml
Note that this has been developed as a proof-of-concept, and may be missing some
features or link to inaccessible Google-internal pages.
## Can I Use This?
If you are a TensorFlow or Google ML Ecosystem project, e.g.
`google/foo`, you can make a PR to add yourself to
https://tensorflow.github.io/build/foo. Simply:
1. Decide on a config name, e.g. `foo`
2. Duplicate an existing configuration yaml file to "foo.yaml" and update it
3. Add `foo` to the matrix in .github/workflows/dashboard.yml
4. Request a review from @angerson and @MichaelHudgins
If you are not a TensorFlow or Google ML Ecosystem project, or if you want to
use your own domain, you can easily host this dashboard yourself. Just fork
this repository and tweak the configuration. You will need to set up GitHub
Pages for the repo and configure it to update from GitHub Actions, then enable
the Dashboard Generator action. In our repo, "tensorflow.yaml" is treated as
the root configuration for the site, so update .github/workflows/dashboard.yml
to use whatever you rename it to instead.
We'd like to one day provide a single GitHub Action you can use instead of
needing to fork this repository.
## My Scheduled GitHub Actions Don't Appear
The dashboard scrapes the Statuses for all branch commits. Scheduled GitHub
Actions don't set statuses by default, but you can update your workflow to
set a commit status explicitly with an Action like [set-commit-status-action](https://github.com/myrotvorets/set-commit-status-action).
Here is a very basic example:
```
jobs:
sets_a_commit:
# Required for set-commit-status-action
permissions:
statuses: write
steps:
- uses: myrotvorets/set-commit-status-action@master
# Always run this even if previous steps fail
if: always()
with:
# Note: defaults to Workflow name. You can set "context" with a
# matrix value if you want to split statuses by their matrix.
status: ${{ job.status }}
# set-commit-status-action doesn't know how to fetch SHA by itself
sha: ${{ github.sha }}
```
## I Lost All My Historical Data, or Want to Change the Dashboard Data
The dashboard accumulates data over time. If you lose it all, have reset it by
accident, or want to delete certain bad data, you can overwrite the data by
creating a GitHub Gist starting from the artifacts from a previous successful
job.
For example, say you want to restore "foo".
1. Download the "foo" artifact data you want to restore from a successful job
2. Create a non-private GitHub Gist containing a file named "foo" whose
contents are the same as "old.json" from the artifact. You can modify the
contents if you wish to delete some data. You can have multiple files if you
want to restore multiple dashboards. Empty or missing files will have no
effect.
3. Trigger the Dashboard Generater workflow and put the Gist ID (a long
hexadecimal string) in the "Overwrite..." field.
## Does this Support PR status monitoring?
No. This may be worked on later but is not currently planned.
#!/usr/bin/env python3
#
# Copyright 2023 The TensorFlow Authors. All Rights Reserved.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
# ============================================================================
#
# Generates the TF OSS dashboard.
# Usage: ./query_api.sh | ./dashboard.py | tee dashboard.html
# You can also do ./query_api.sh > data.json and then do:
# cat data.json | ./dashboard.py | tee dashboard.html
# Usage: ./query_api.sh config.yaml | ./dashboard.py config.yaml | tee dashboard.html
# You can also do ./query_api.sh config.yaml > data.json and then do:
# cat data.json | ./dashboard.py config.yaml | tee dashboard.html
from collections import defaultdict
from jinja2 import Environment, FileSystemLoader
import arrow
import cmarkgfm
import itertools
import json
import cmarkgfm
import os.path
import pypugjs
import re
import subprocess
import sys
import yaml
with open('config.yaml', 'r') as f:
OUTDIR=sys.argv[1]
with open(sys.argv[1] + ".yaml", 'r') as f:
YAML_CONFIG = yaml.safe_load(f)
JSON_DATA = json.load(sys.stdin)
......@@ -46,7 +64,7 @@ JSON_DATA = json.load(sys.stdin)
# plus all its associated commit metadata.
all_records = []
CHANGELIST_REGEX = re.compile(r"PiperOrigin-RevId: (\d+)")
for commit in JSON_DATA["data"]["repository"]["defaultBranchRef"]["target"]["history"]["nodes"]:
for commit in JSON_DATA["data"]["repository"]["ref"]["target"]["history"]["nodes"]:
# Ignore commits with no statusCheckRollup, which can happen sometimes --
# maybe when a commit has no results at all yet for any jobs?
if commit["statusCheckRollup"] is None:
......@@ -82,10 +100,7 @@ for commit in JSON_DATA["data"]["repository"]["defaultBranchRef"]["target"]["his
clone["type"] = "status check"
clone["state"] = check["state"]
clone["result_url"] = check["targetUrl"]
# Right now we treat any URL that does not have "http://fusion" in it
# as a public URL: this is a Google-only URL target that points to an
# internal review system that outsiders can't see.
clone["is_public"] = "http://fusion" not in check["targetUrl"]
clone["is_public"] = not check["targetUrl"].startswith(tuple(YAML_CONFIG["internal_startswith"]))
# GitHub Actions jobs are all other kinds of jobs.
else:
# Some GitHub Actions results don't have a workflow group name, so we
......@@ -101,6 +116,8 @@ for commit in JSON_DATA["data"]["repository"]["defaultBranchRef"]["target"]["his
clone["state"] = check["conclusion"] or check["status"]
clone["result_url"] = check["url"]
clone["is_public"] = True
if not YAML_CONFIG["internal_shown"] and not clone["is_public"]:
continue
if clone["name"] in YAML_CONFIG["hidden"]:
continue
all_records.append(clone)
......@@ -122,6 +139,7 @@ for record in all_records:
# included in the nightly jobs if it was committed before a nightly job's
# commit. This way we can quickly tell when a commit was first tested in a TF
# Nightly test.
if YAML_CONFIG["nightly_job_basis"]:
nightlies = job_names_to_records[YAML_CONFIG["nightly_job_basis"]]
for name, records in job_names_to_records.items():
for record in records:
......@@ -135,7 +153,11 @@ for name, records in job_names_to_records.items():
# two job results by comparing the commit hash between two neighboring records.
for job_name, original_records in job_names_to_records.items():
for later, earlier in itertools.pairwise(original_records):
later["previous_diff_url"] = f"https://github.com/tensorflow/tensorflow/compare/{earlier['commit']}...{later['commit']}"
later["previous_diff_url"] = "https://github.com/{}/{}/compare/{}...{}".format(
YAML_CONFIG["repo_owner"],
YAML_CONFIG["repo_name"],
earlier['commit'],
later['commit'])
# Now that all the individual record preprocessing is done, every record has
# everything it needs to be displayed individually as part of a commit record.
......@@ -219,7 +241,7 @@ for category, job_names in YAML_CONFIG["categories"].items():
# Finally, pass everything to the template and render it
with open("help.md", "r") as f:
helptext = cmarkgfm.github_flavored_markdown_to_html(f.read())
helptext = cmarkgfm.github_flavored_markdown_to_html(YAML_CONFIG["help"] + "\n\n" + f.read())
env = Environment(
loader=FileSystemLoader('.'),
extensions=['pypugjs.ext.jinja.PyPugJSExtension']
......@@ -227,7 +249,8 @@ env = Environment(
template = env.get_template('template.html.pug')
now = arrow.now().to('US/Pacific').format("ddd, MMM D [at] h:mma ZZZ")
isonow = arrow.now().to('US/Pacific').isoformat()
print(template.render(
with open(os.path.join(OUTDIR, "index.html"), "w") as f:
f.write(template.render(
by_group=by_group,
by_commit=commits_to_records,
helptext=helptext,
......@@ -245,4 +268,4 @@ for category in YAML_CONFIG["badges"]:
url = f"https://img.shields.io/static/v1?label={category}&message={passed} passed, 0 failed&color=success"
else:
url = f"https://img.shields.io/static/v1?label={category}&message={passed} passed, {failed} failed&color=critical"
subprocess.run(["wget", url, "-O", f"{category}.svg"])
subprocess.run(["wget", url, "-O", f"{OUTDIR}/{category}.svg"])
This is TensorFlow's open-source build status dashboard. It tracks all
GitHub statuses for the TensorFlow repository that are published to GitHub.
The source for the dashboard is on [TensorFlow SIG Build](https://github.com/tensorflow/build/tree/master/tf_oss_dashboard).
Many of these jobs use Google's internal continuous integration systems, and may
not report their results publicly. We're trying to make more of our important
jobs visible to external developers, but security concerns make this a slow
process.
Here are some tips and notes about the dashboard:
The source for this dashboard is on [TensorFlow SIG
Build](https://github.com/tensorflow/build/tree/master/tf_oss_dashboard). Here
are some tips and notes about the dashboard:
#### Basic Usage
- This page refreshes roughly every 5 minutes to check for updates.
- Click on a status dot to see all statuses for that commit. The job you
clicked is highlighted.
- Click the left toggle switch in the navbar to show a section at the top
......@@ -23,7 +17,6 @@ Here are some tips and notes about the dashboard:
easy way to check if your CL has landed on GitHub, on Nightlies, etc). You
can include the "cl/" too if you're copy-pasting.
- Add `#<pr-number>` to the dashboard URL to find a specific merged PR.
- This page refreshes roughly every 5 minutes to check for updates.
- The last dot you clicked on is outlined in black. It goes away after refresh.
- Click the "Reveal All" button in a modal to highlight every dot for that
commit. This is useful for e.g. seeing all commits after a nightly release.
......@@ -51,20 +44,15 @@ Here are some tips and notes about the dashboard:
commit for that job. GitHub occasionally reports that there is actually no
diff. Take a screenshot and let us know if that happens.
#### Surprises
#### Miscellaneous
- Jobs that run more than once on the same commit have different status
dots but get doubled-up in the commit overview. This isn't very common.
- Most nightly jobs run on the same commit, but some don't.
- The date on nightly jobs is the date of the final commit that was included in
that job. The TF team sometimes refers to these in a different way: the
"Nightly for today" usually refers to the Nightly jobs whose final commits
were *yesterday*. So if today is Feb 11, you want the Nightly labeled Feb 10.
- Jobs that run more than once on the same commit have different status
dots but get doubled-up in the commit overview. This isn't very common.
- Jobs which ran on a PR that was cleanly merged also appear on this dashboard.
These commits do not show CLs for internal users. For example,
`import/copybara` is a pull-request job whose status also appears on its
merge commit, if Copybara decides to merge the PR directly. It's unknown
whether this happens to all PRs or not.
- Kokoro, Google's CI system that powers most of these jobs, does not report
"in progress" jobs, so there is no way to see how many Kokoro jobs are
pending.
......
#!/usr/bin/env python3
#
# Copyright 2023 The TensorFlow Authors. All Rights Reserved.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
# ============================================================================
#
# Merge two GitHub GraphQL response data JSON files.
#
# Usage:
# merge.py config.yaml old.json new.json > merged.json
import json
import sys
import yaml
with open(sys.argv[1], 'r') as f:
YAML_CONFIG = yaml.safe_load(f)
with open(sys.argv[1], "r") as f:
old = json.load(f)
with open(sys.argv[2], "r") as f:
old = json.load(f)
with open(sys.argv[3], "r") as f:
new = json.load(f)
overlap = False
commits = {}
for commit in old["data"]["repository"]["defaultBranchRef"]["target"]["history"]["nodes"]:
for commit in old["data"]["repository"]["ref"]["target"]["history"]["nodes"]:
commits[commit["oid"]] = commit
for commit in new["data"]["repository"]["defaultBranchRef"]["target"]["history"]["nodes"]:
for commit in new["data"]["repository"]["ref"]["target"]["history"]["nodes"]:
if commit["oid"] in commits:
overlap = True
commits[commit["oid"]] = commit
......@@ -21,8 +45,9 @@ print("Total number of commits:", len(commits), file=sys.stderr)
if overlap:
a = list(commits.values())
a.sort(key=lambda x: x["committedDate"])
# Only store the last 1000 commits worth of data, which is roughly 2 wks max
# The sort is ascending (-2 days..yesterday..today), so [-1000:] gets the
# 1000 most recent commits.
new["data"]["repository"]["defaultBranchRef"]["target"]["history"]["nodes"] = a[-1000:]
# Only store the last N commits worth of data, based on history_size from the
# yaml config. For TF it's 1000, which is roughly 2 wks max. The sort is
# ascending (-2 days..yesterday..today), so [-1000:] gets the 1000 most recent
# commits.
new["data"]["repository"]["ref"]["target"]["history"]["nodes"] = a[-YAML_CONFIG["history_size"]:]
print(json.dumps(new))
#!/bin/bash
# Copyright 2023 The TensorFlow Authors. All Rights Reserved.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
# ============================================================================
#
# Usage:
# merge_and_generate.sh [name of config file]
# e.g. merge_and_generate.sh jax
set -euxo pipefail
pip install -r requirements.txt
NAME=$1
mkdir -p $NAME
./query_api.sh $NAME.yaml $NAME/new.json
echo "::group::new data (LARGE)"
cat $NAME/new.json
echo "::endgroup::"
if [[ -e "$NAME/old.json" ]]; then
echo "::group::old data (LARGE)"
cat $NAME/old.json
echo "::endgroup::"
echo "::group::Merged data"
./merge.py $NAME.yaml $NAME/old.json $NAME/new.json | tee $NAME/merged.json
echo "::endgroup::"
else
mv $NAME/new.json $NAME/merged.json
fi
echo "::group::Dashboard"
cat $NAME/merged.json | ./dashboard.py $NAME
echo "::endgroup::"
mv $NAME/merged.json $NAME/old.json
{
repository(name: "tensorflow", owner: "tensorflow") {
defaultBranchRef {
query($name:String!, $owner:String!, $branch:String!) {
repository(name: $name, owner: $owner) {
ref(qualifiedName: $branch) {
target {
... on Commit {
history(first: 100) {
......
#!/usr/bin/env bash
#
# Copyright 2023 The TensorFlow Authors. All Rights Reserved.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
# ============================================================================
#
# Usage:
# export GITHUB_TOKEN=[your github token, see github.com/settings/tokens]
# query_api.sh config.yaml > dashboard_data.json
#
# Send the dashboard data graphql query (query.graphql) to the GitHub GraphQL
# API, choosing the owner and repository (e.g. tensorflow/tensorflow) from the
# provided config file. You must have a valid GITHUB_TOKEN in the env.
#
# Requires "yq" and "jq"
#
# See https://docs.github.com/en/graphql/overview/explorer
curl -s https://api.github.com/graphql -X POST \
-H "Authorization: Bearer $GITHUB_TOKEN" \
-H "Content-Type: application/json" \
-d "$(jq -c -n --arg query "$(cat query.graphql)" '{"query":$query}')"
# Convert config yaml file to json
echo "::group::config.json"
yq -o json . $1 | tee config.json || exit 0
echo "::endgroup::"
# <<'EOF' is a quoted heredoc that allows literal $ signs.
# See https://www.gnu.org/savannah-checkouts/gnu/bash/manual/bash.html#Here-Documents
# Note that in query.graphql, query($name:String!, $owner:String!) declare that
# those two variables will come along with the query. Format is $name:Type!
echo "::group::script.jq"
tee script.jq <<'EOF'
{
query: $query,
variables: {
owner: .repo_owner,
name: .repo_name,
branch: .repo_branch
}
}
EOF
echo "::endgroup::"
# Generate a well-formed and escaped JSON payload that contains the graphQL
# query and its variables. jq loads the variables for us and also escapes any
# weird symbols.
echo "::group::query.json"
jq --rawfile query query.graphql --from-file script.jq config.json | tee query.json
echo "::endgroup::"
# Note that curl accepts "raw data" or @filename for the --data flag
echo "::group::curl call"
curl https://api.github.com/graphql --request POST \
--header "Authorization: Bearer $GITHUB_TOKEN" \
--header "Content-Type: application/json" \
--data @query.json | tee $2
echo "::endgroup::"
// Copyright 2023 The TensorFlow Authors. All Rights Reserved.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
// ============================================================================
//
// Display warning banner if use of analytics cookies hasn't been acknowledged
if (location.hostname !== "" && !Cookies.get("tf-cookies-accepted")) {
$(".tf-cookie-warning").removeClass("d-none")
......@@ -115,6 +130,14 @@ $(function () {
unescaped = decodeURIComponent(window.location.hash).replace("#", "").replace("+", "")
if (window.location.hash.length <= 1) {
// Nothing to do if no hash in the URL
// If the hash is exactly 41 chars (hash sign # plus a 40-char sha hash),
// just show that modal.
} else if (window.location.hash.length == 41) {
if ($(window.location.hash).length) {
new bootstrap.Modal(window.location.hash).show()
} else {
new bootstrap.Modal("#tf-no-commit-modal").show()
}
// If the hash matches a Category on the page, then scroll to it.
} else if (document.getElementById(unescaped)) {
// Google Chrome, at least, does not scroll if the tab is opened in the
......@@ -133,14 +156,6 @@ $(function () {
}
})
}
// If the hash is exactly 41 chars (hash sign # plus a 40-char sha hash),
// just show that modal.
} else if (window.location.hash.length == 41) {
if ($(window.location.hash).length) {
new bootstrap.Modal(window.location.hash).show()
} else {
new bootstrap.Modal("#tf-no-commit-modal").show()
}
// And if it's not a commit sha, it's either a PR or a CL, so try and find a
// modal matching that PR number if the length is short (CLs will always
// be nine or more characters)
......
#!/bin/bash
set -euxo pipefail
pip install -r requirements.txt
./query_api.sh > new.json
echo "::group::New Query Data"
cat new.json
echo "::endgroup::"
if [[ -e "../Merged Data/old.json" ]]; then
mv "../Merged Data/old.json" old.json
./merge.py old.json new.json > merged.json
echo "::group::Merged Data"
cat merged.json
echo "::endgroup::"
else
mv new.json merged.json
fi
echo "::group::Dashboard Generator Output"
cat merged.json | ./dashboard.py | tee index.html
echo "::endgroup::"
mv merged.json old.json
/* Copyright 2023 The TensorFlow Authors. All Rights Reserved.
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
============================================================================
*/
/* =============================== */
/* GENERAL */
/* =============================== */
......@@ -32,30 +47,6 @@ body {
display: block;
}
/* Cyclical auto-refresh timer */
/* Works in tandem with javascript. The 300s CSS animation aligns with the */
/* auto-refresh timer in script.js. */
@property --timer-percentage {
initial-value: 100%;
inherits: false;
syntax: "<percentage>";
}
.tf-refresh-timer {
background: conic-gradient(white var(--timer-percentage), rgba(0,0,0,0) 0);
border-radius: 50%;
width: 0.75em;
height: 0.75em;
animation: tf-refresh-timer 300s 1 linear forwards;
display: inline-block;
}
@keyframes tf-refresh-timer {
to {
--timer-percentage: 0%;
}
}
/* =============================== */
/* JOB CARDS */
/* Note that we're layering this config on top of bootstrap's card class */
......
//- Copyright 2023 The TensorFlow Authors. All Rights Reserved.
//-
//- Licensed under the Apache License, Version 2.0 (the "License");
//- you may not use this file except in compliance with the License.
//- You may obtain a copy of the License at
//-
//- http://www.apache.org/licenses/LICENSE-2.0
//-
//- Unless required by applicable law or agreed to in writing, software
//- distributed under the License is distributed on an "AS IS" BASIS,
//- WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
//- See the License for the specific language governing permissions and
//- limitations under the License.
//- ============================================================================
!!! 5
html(lang="en")
head
title TensorFlow GitHub Status
title= yaml["title"]
meta(charset="UTF-8")
link(rel="icon" href="favicon.png")
link(rel="icon" href="/favicon.png")
link(href="https://cdn.jsdelivr.net/npm/bootstrap@5.3.0-alpha1/dist/css/bootstrap.min.css" rel="stylesheet" integrity="sha384-GLhlTQ8iRABdZLl6O3oVMWSktQOp6b7In1Zl3/Jr59b6EGGoI1aFkw7cmDA6j6gD" crossorigin="anonymous")
script(src="https://cdn.jsdelivr.net/npm/bootstrap@5.3.0-alpha1/dist/js/bootstrap.bundle.min.js" integrity="sha384-w76AqPfDkMBDXo30jS1Sgez6pr3x5MlQ1ZAGC+nuZB+EYdgRZgiwxhTBTkF7CXvN" crossorigin="anonymous")
script(src="https://code.jquery.com/jquery-3.6.4.slim.min.js" integrity="sha256-a2yjHM4jnF9f54xUQakjZGaqYs/V1CYvWpoqZzC2/Bw=" crossorigin="anonymous")
script(src="https://cdn.jsdelivr.net/npm/js-cookie@3.0.5/dist/js.cookie.min.js")
script(src="https://cdnjs.cloudflare.com/ajax/libs/moment.js/2.29.3/moment.min.js" crossorigin="anonymous" referrerpolicy="no-referrer")
link(rel="stylesheet" href="style.css")
link(rel="stylesheet" href="/build/style.css")
//- Google Analytics. Conditionally enabled if "gtag" is set in config.yaml
if yaml["gtag"]:
script(async src="https://www.googletagmanager.com/gtag/js?id=#{yaml['gtag']}")
......@@ -19,24 +33,25 @@ html(lang="en")
gtag('js', new Date());
gtag('config', "#{yaml['gtag']}");
body
if yaml["gtag"]
.tf-cookie-warning.p-2.bg-warning.navbar-text.d-none
.text-center The TensorFlow GitHub CI Dashboard uses cookies from Google to analyze traffic.
.text-center The #{yaml["title"]} Dashboard uses cookies from Google to analyze traffic.
a(href="https://policies.google.com/technologies/cookies") Learn more.
a.fw-bold.ps-4#tf-accept-cookies(role="button") Ok, got it. Hide this message.
nav.navbar.navbar-dark.navbar-expand-lg.py-0.mb-2
.container-fluid.py-0
a(href=).navbar-brand TensorFlow GitHub CI
a(href=).navbar-brand= yaml["title"]
.align-self-start.navbar-text#tf-now(data-isonow=isonow) Updated
a(title="Check the dashboard deployment GitHub Actions workflow" href="https://github.com/tensorflow/build/actions/workflows/dashboard.yml") #{now}
|
a#tf-ago
.ps-1.tf-refresh-timer(title="The page auto-refreshes every 5 minutes (delayed if a modal is open) to check and see if the dashboard has been updated.")
.flex-grow-1
button.navbar-toggler(type="button" data-bs-toggle="collapse" data-bs-target="#navbarToggle" aria-controls="navbarTogglerDemo02" aria-expanded="false" aria-label="Toggle navigation")
span.navbar-toggler-icon
.collapse.navbar-collapse#navbarToggle
ul.navbar-nav.ms-auto
if yaml["buildcop"]
.form-check.form-switch.m-auto
input.form-check-input(title="Toggle failure section" type="checkbox" role="switch" id="tf-show-buildcop")
.form-check.form-switch.m-auto
......@@ -44,12 +59,12 @@ html(lang="en")
li.nav-item
a.nav-link(role="button" data-bs-toggle="modal" data-bs-target="#tf-help-modal") Help
li.nav-item
a.nav-link(href="https://github.com/tensorflow/tensorflow/commits/master") Commits
a.nav-link(href="https://github.com/#{yaml['repo_owner']}/#{yaml['repo_name']}/commits/#{yaml['repo_branch']}") Commits
li.nav-item
a.nav-link(href="https://github.com/tensorflow/build/issues/142") Suggestions
//- putting this here means there's usually no flash when the page reloads
script(src="script.js")
script(src="/build/script.js")
.d-flex.flex-column.gap-1.m-1
.tf-failures-section
......@@ -64,6 +79,7 @@ html(lang="en")
.card-body
h5.text-center.fw-bold= name
.d-flex.flex-row.flex-wrap.gap-1
if yaml["internal_shown"]
if tests[0]["is_public"]
.badge.p-2.tf-public(title="This job has publicly-visible results pages.") Public
else
......@@ -82,6 +98,7 @@ html(lang="en")
.card-body
h5.text-center.fw-bold= name
.d-flex.flex-row.flex-wrap.gap-1
if yaml["internal_shown"]
if tests[0]["is_public"]
.badge.p-2.tf-public(title="This job has publicly-visible results pages.") Public
else
......@@ -104,6 +121,7 @@ html(lang="en")
a.badge.tf-cl(href=jobs[0]["cl_url"]) cl/#{jobs[0]["cl"]}
else
span.badge.tf-nocl No CL Attached
if yaml["nightly_job_basis"]
if "first_nightly_sha" not in jobs[0]:
.badge.bg-primary No Nightly Yet
elif jobs[0]["first_nightly_sha"] == jobs[0]["commit_id"]:
......@@ -117,6 +135,7 @@ html(lang="en")
table.table.table-sm.table-striped
each job in jobs
tr.py-1.px-1.m-0.text-center.align-middle
if yaml["internal_shown"]
td
if job["is_public"]
span.badge.tf-public Public
......@@ -143,14 +162,14 @@ html(lang="en")
.modal-dialog.modal-dialog-centered
.modal-content
.modal-header.fw-bold CL Not Found
.modal-body Sorry, that CL isn't on the dashboard. If new, it may not be on GitHub yet, or may not have any CI results yet. If it's an older CL, it may be old enough that the data window no longer includes it. The dashboard only displays the last one thousand TF commits, which is usually about two weeks of commits.
.modal-body Sorry, that CL isn't on the dashboard. If new, it may not be on GitHub yet, or may not have any CI results yet. If it's an older CL, it may be old enough that the data window no longer includes it. The dashboard only displays the last #{yaml["history_size"]} commits; for TensorFlow, 1000 is usually about two weeks of commits.
.modal.modal-lg.fade(tabindex='-1' aria-labelledby="tf-no-commit-modal" id="tf-no-commit-modal" aria-hidden='true')
.modal-dialog.modal-dialog-centered
.modal-content
.modal-header.fw-bold Commit Not Found
.modal-body Sorry, that commit isn't on the dashboard. If new, it may not have any CI results yet. If it's an older commit, it may be old enough that the data window no longer includes it. The dashboard only displays the last one thousand TF commits, which is usually about two weeks of commits.
.modal-body Sorry, that commit isn't on the dashboard. If new, it may not be on GitHub yet, or may not have any CI results yet. If it's an older CL, it may be old enough that the data window no longer includes it. The dashboard only displays the last #{yaml["history_size"]} commits; for TensorFlow, 1000 is usually about two weeks of commits.
.modal.modal-lg.fade(tabindex='-1' aria-labelledby="tf-no-pr-modal" id="tf-no-pr-modal" aria-hidden='true')
.modal-dialog.modal-dialog-centered
.modal-content
.modal-header.fw-bold PR Not Found
.modal-body Sorry, that PR isn't on the dashboard. If new, it may not have any CI results yet, or may not be merged yet. If it's an older PR, it may be old enough that the data window no longer includes it. The dashboard only displays the last one thousand TF commits, which is usually about two weeks of commits.
.modal-body Sorry, that PR isn't on the dashboard. If new, it may not be on GitHub yet, or may not have any CI results yet. If it's an older CL, it may be old enough that the data window no longer includes it. The dashboard only displays the last #{yaml["history_size"]} commits; for TensorFlow, 1000 is usually about two weeks of commits.
# Copyright 2023 The TensorFlow Authors. All Rights Reserved.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
# ============================================================================
title: TensorFlow GitHub CI Status
repo_owner: tensorflow
repo_name: tensorflow
repo_branch: master
help: >-
This is TensorFlow's open-source build status dashboard. It tracks all GitHub
statuses for the TensorFlow repository that are published to GitHub.
Many of these jobs use Google's internal continuous integration systems, and
may not report their results publicly. We're trying to make more of our
important jobs visible to external developers, but security concerns make this
a slow process.
categories:
TF Official Continuous:
- Py+CPP Test Suite - Ubuntu GPU, Python 3.9
......@@ -46,11 +72,15 @@ buildcop:
- TF Official Nightly
short_sha_length: 7
default_category: Everything Else
# Use this to determine which commits are the Nightly ones when comparing
# Use this to determine which commits are the Nightly ones when comparing.
# Set it to false to disable all "Nightly" features (i.e. the "in nightly..."
# or "this is nightly" badge in a commit)
nightly_job_basis: Nightly - Code Check - Linux
# Configures how large a job card can grow (that is, how many dots will appear
# on it). A date tag is size 3, and a status dot is size 1.
maximum_card_size: 100
# How many commits to keep in the history
history_size: 1000
# Set to Google Analytics "Measurement ID" to enable, or disable w/ "false"
gtag: G-JTD613F3TX
# Generate SVG badges, each available at Category Name.svg, e.g.
......@@ -59,3 +89,9 @@ gtag: G-JTD613F3TX
badges:
- TF Official Continuous
- TF Official Nightly
# Internal jobs are those whose result URL starts with...
internal_startswith:
- "http://fusion"
- "http://cl/"
# Show internal-only jobs. If false, hides the public/private indicators
internal_shown: true
0% Loading or .
You are about to add 0 people to the discussion. Proceed with caution.
Please register or to comment