Skip to content
Snippets Groups Projects
Unverified Commit a54d1edf authored by Austin Anderson's avatar Austin Anderson Committed by GitHub
Browse files

Rearrange files and projects to match #16 (#17)

:tada: 
parent 634242dd
Branches
No related tags found
No related merge requests found
Showing
with 119 additions and 93 deletions
## Expected Behavior
## Actual Behavior
## Steps to Reproduce the Problem
1.
1.
1.
## Specifications
- Version:
- Platform:
\ No newline at end of file
Fixes #<issue_number_goes_here>
> It's a good idea to open an issue first for discussion.
- [ ] Tests pass
- [ ] Appropriate changes to README are included in PR
\ No newline at end of file
# SIG Leads are default for PRs unless a specific owner is listed below
* @angerson @perfinion
# These should match the Maintainer: line in your README.md
# Please sort them in alphabetical order.
/directory_template @angerson @perfinion
/manylinux_2014_docker_images @sub-mod
/ppc64le @wdirons
/tekton @perfinion
# How to Contribute # How to Contribute
We'd love to accept your patches and contributions to this project. There are To add your own project to our showcase, you can:
just a few small guidelines you need to follow.
* Add a link and a short description to [README.md](README.md) in the same
format as the other links, and then file a PR, or,
* Copy the [Directory Template](directory_template) into a new folder, fill in
the template, and add it to the showcase. Make sure to update
[CODEOWNERS](CODEOWNERS) as well.
This repo is a good place for small additions, but if the project is a big one,
it may be better to maintain it in your own repository for better encapsulation
of responsibility (and so you can make your own rules for further
contributions).
There are just a few small standard guidelines you need to follow as well:
## Contributor License Agreement ## Contributor License Agreement
......
...@@ -7,22 +7,49 @@ ...@@ -7,22 +7,49 @@
[![Gitter chat](https://img.shields.io/badge/chat-on%20gitter-46bc99.svg)](https://gitter.im/tensorflow/sig-build) [![Gitter chat](https://img.shields.io/badge/chat-on%20gitter-46bc99.svg)](https://gitter.im/tensorflow/sig-build)
[![SIG Build Forum](https://img.shields.io/badge/discuss-on%20tensorflow.org-orange)](https://groups.google.com/a/tensorflow.org/g/build) [![SIG Build Forum](https://img.shields.io/badge/discuss-on%20tensorflow.org-orange)](https://groups.google.com/a/tensorflow.org/g/build)
**TensorFlow SIG Build** is a community group dedicated to improving all aspects **TensorFlow SIG Build** is a community group dedicated to the TensorFlow build
of the TensorFlow build process. process. This repository is a showcase of resources, guides, tools, and builds
contributed by the community, for the community.
## Contributing ## Group
SIG Build is a community led open source project. As such, the project ### Contributing
SIG Build is a community-led open source project. As such, the project
depends on public contributions, bug-fixes, and documentation. Please depends on public contributions, bug-fixes, and documentation. Please
see [contribution guidelines](CONTRIBUTING.md) for a guide on how to see [contribution guidelines](CONTRIBUTING.md) for a guide on how to
contribute. This project adheres to [TensorFlow's code of conduct](CODE_OF_CONDUCT.md). contribute. This project adheres to [TensorFlow's code of conduct](CODE_OF_CONDUCT.md).
By participating, you are expected to uphold this code. By participating, you are expected to uphold this code.
## Community ### Community
* [Public Mailing List](https://groups.google.com/a/tensorflow.org/forum/#!forum/build) * [Public Mailing List](https://groups.google.com/a/tensorflow.org/forum/#!forum/build)
* [SIG Monthly Meeting Notes](https://docs.google.com/document/d/10_3IQ5aF-88ADJNLF0WOpb09bZ15x-sBnRSnDHNCNr8/edit) * [SIG Monthly Meeting Notes](https://docs.google.com/document/d/10_3IQ5aF-88ADJNLF0WOpb09bZ15x-sBnRSnDHNCNr8/edit)
* Join our mailing list and receive calendar invites to the meeting. * Join our mailing list and receive calendar invites to the meeting.
## License ### License
[Apache License 2.0](LICENSE) [Apache License 2.0](LICENSE)
## Project Showcase
Want to add your own project to this list? It's easy: check out
[CONTRIBUTING.md](CONTRIBUTING.md).
### Docker
* [**Manylinux 2014 Docker Images**](manylinux_2014_docker_images):
`manylinux2014` build environment for TensorFlow packages.
* [**Distroless Dockerfiles**](https://github.com/uvarc/rivanna-docker):
Distroless ([info](https://github.com/GoogleContainerTools)) TensorFlow
images, which are smaller than TensorFlow's official images.
### Platforms
* [**ppc64le Builds**](ppc64le_builds): Dockerfiles and wheel build scripts for
building TF on ppc64le.
### WIP / Other
* [**Tekton CI**](tekton): perfinion's experimental directory for using Tekton
CI with TensorFlow
* [**Directory Template**](directory_template): Example short description.
# Directory Template
Example short description.
Maintainer: @angerson (TensorFlow, SIG Build)
* * *
This is is a sample of a new directory for showcasing a local project. Anything
goes, just please follow these rules to keep things straightforward:
- Name the directory the same as the title `in_snake_case` (all lowercase, with
underscores, no hyphens).
- Your directory should have a README.md with a header:
- Title identical to the one added on the root README.md
- Short description identical to the one added on the root README.md
- Maintainer: `<your email(s)> (<your organization(s)>)`. If you have no
organization or not acting on behalf of one, write "Independent".
- You can add this directory to the root README.md list under any Showcase
section. If it doesn't fit under any section, just put it at the top under
"Showcase."
FROM ubuntu:16.04
LABEL maintainer="Jason Zaman <jason@perfinion.com>"
# Copy and run each install script one at a time for better layer caching.
COPY install/install_deb_packages.sh /install/
RUN /install/install_deb_packages.sh
COPY install/install_golang.sh /install/
RUN /install/install_golang.sh
COPY install/install_proto3.sh /install/
RUN /install/install_proto3.sh
COPY install/install_buildifier.sh /install/
RUN /install/install_buildifier.sh
COPY install/install_bazel.sh /install/
RUN /install/install_bazel.sh
COPY install/install_pip_packages.sh /install/
RUN /install/install_pip_packages.sh
# Set up the master bazelrc configuration file.
COPY install/bazelrc /etc/bazel.bazelrc
# Dockerfiles for TensorFlow builds
## manylinux2014 builds
Run the below commands to create manylinux2014 build environment.
```
docker build -t build/manylinux2014 -f Dockerfile.manylinux2014.cpu .
docker run -it build/manylinux2014:latest /bin/bash
```
Run the below commands to build TF.
```
source scl_source enable devtoolset-7 rh-python36
git clone --branch=r2.1 --depth=1 https://github.com/tensorflow/tensorflow.git
cd tensorflow/
./configure
bazel build -c opt --cxxopt="-D_GLIBCXX_USE_CXX11_ABI=0" --verbose_failures //tensorflow/tools/pip_package:build_pip_package
```
## Distroless Images
The [Distroless](https://github.com/GoogleContainerTools/distroless) base images developed by Google are meant to contain an application and its runtime dependencies only. Please visit their GitHub for more information.
Distroless Dockerfiles for TensorFlow CPU (`Dockerfile.cpu`) and GPU (`Dockerfile.distroless`) are hosted in the [rivanna-docker repository](https://github.com/uvarc/rivanna-docker). Using the [MNIST benchmark](https://www.tensorflow.org/tutorials/quickstart/beginner), the containers were tested on UVA's HPC Rivanna cluster to have the same performance as the official ones.
Image size comparison:
| [Official](https://hub.docker.com/r/tensorflow/tensorflow/tags) | Size | [Distroless](https://hub.docker.com/r/uvarc/tensorflow/tags) | Size | Reduction |
|---|---|---|---|---|
| `2.3.0` | 582 MB | `2.3.0-cpu` | 227 MB | 61% |
| `2.3.0-gpu` | 1.44 GB | `2.3.0-distroless` | 1.18 GB | 18% |
| `nightly-gpu` | 2.27 GB | `2.4.0-distroless` | 1.47 GB | 35% |
Pull command:
```
docker pull uvarc/tensorflow:<tag>
```
### Contact
Research Computing at the University of Virginia (external) maintains Distroless Dockerfiles for TensorFlow in the [rivanna-docker repository](https://github.com/uvarc/rivanna-docker). Please reach out to hpc-support@virginia.edu for questions and feedback.
# Manylinux2014 Docker Images
`manylinux2014` build environment for TensorFlow packages.
Maintainer: @sub-mod (Red Hat)
* * *
Run the below commands to create a manylinux2014 build environment.
```
docker build -t build/manylinux2014 -f Dockerfile.manylinux2014.cpu .
docker run -it build/manylinux2014:latest /bin/bash
```
Run the below commands to build TF.
```
source scl_source enable devtoolset-7 rh-python36
git clone --branch=r2.1 --depth=1 https://github.com/tensorflow/tensorflow.git
cd tensorflow/
./configure
bazel build -c opt --cxxopt="-D_GLIBCXX_USE_CXX11_ABI=0" --verbose_failures //tensorflow/tools/pip_package:build_pip_package
```
# ppc64le Builds
Dockerfiles and wheel build scripts for building TF on ppc64le.
Maintainer: @wdirons (IBM)
* * *
`cpu/Dockerfile.manylinux_2014` extends from the
quay.io/pypa/manylinux2014_ppc64le docker image and installs java, bazel (from
source), and the minimal required pip packages to build TensorFlow. It also adds
the jenkins userid necessary to work with Oregon State University's Open Source
Lab jenkins environment. `gpu/Dockerfile.manylinux_2014.cuda10_1` extends from
the cpu image and adds CUDA 10.1 and cuDNN 7.6.
The bash scripts are the equivalent of what is run in the TensorFlow builds done
with Jenkins (everything except the remote cache server). They can also be run
manually by cloning tensorflow and copying the script into the root of the
workspace.
0% Loading or .
You are about to add 0 people to the discussion. Proceed with caution.
Please register or to comment