initial commit

Signed-off-by: Sean Cross <sean@xobs.io>
This commit is contained in:
Sean Cross 2018-09-27 15:34:24 +08:00
commit 2550876426
3 changed files with 667 additions and 0 deletions

190
LICENSE Normal file
View File

@ -0,0 +1,190 @@
Apache License
Version 2.0, January 2004
http://www.apache.org/licenses/
Copyright 2018 Sean Cross
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION
1. Definitions.
"License" shall mean the terms and conditions for use, reproduction,
and distribution as defined by Sections 1 through 9 of this document.
"Licensor" shall mean the copyright owner or entity authorized by
the copyright owner that is granting the License.
"Legal Entity" shall mean the union of the acting entity and all
other entities that control, are controlled by, or are under common
control with that entity. For the purposes of this definition,
"control" means (i) the power, direct or indirect, to cause the
direction or management of such entity, whether by contract or
otherwise, or (ii) ownership of fifty percent (50%) or more of the
outstanding shares, or (iii) beneficial ownership of such entity.
"You" (or "Your") shall mean an individual or Legal Entity
exercising permissions granted by this License.
"Source" form shall mean the preferred form for making modifications,
including but not limited to software source code, documentation
source, and configuration files.
"Object" form shall mean any form resulting from mechanical
transformation or translation of a Source form, including but
not limited to compiled object code, generated documentation,
and conversions to other media types.
"Work" shall mean the work of authorship, whether in Source or
Object form, made available under the License, as indicated by a
copyright notice that is included in or attached to the work
(an example is provided in the Appendix below).
"Derivative Works" shall mean any work, whether in Source or Object
form, that is based on (or derived from) the Work and for which the
editorial revisions, annotations, elaborations, or other modifications
represent, as a whole, an original work of authorship. For the purposes
of this License, Derivative Works shall not include works that remain
separable from, or merely link (or bind by name) to the interfaces of,
the Work and Derivative Works thereof.
"Contribution" shall mean any work of authorship, including
the original version of the Work and any modifications or additions
to that Work or Derivative Works thereof, that is intentionally
submitted to Licensor for inclusion in the Work by the copyright owner
or by an individual or Legal Entity authorized to submit on behalf of
the copyright owner. For the purposes of this definition, "submitted"
means any form of electronic, verbal, or written communication sent
to the Licensor or its representatives, including but not limited to
communication on electronic mailing lists, source code control systems,
and issue tracking systems that are managed by, or on behalf of, the
Licensor for the purpose of discussing and improving the Work, but
excluding communication that is conspicuously marked or otherwise
designated in writing by the copyright owner as "Not a Contribution."
"Contributor" shall mean Licensor and any individual or Legal Entity
on behalf of whom a Contribution has been received by Licensor and
subsequently incorporated within the Work.
2. Grant of Copyright License. Subject to the terms and conditions of
this License, each Contributor hereby grants to You a perpetual,
worldwide, non-exclusive, no-charge, royalty-free, irrevocable
copyright license to reproduce, prepare Derivative Works of,
publicly display, publicly perform, sublicense, and distribute the
Work and such Derivative Works in Source or Object form.
3. Grant of Patent License. Subject to the terms and conditions of
this License, each Contributor hereby grants to You a perpetual,
worldwide, non-exclusive, no-charge, royalty-free, irrevocable
(except as stated in this section) patent license to make, have made,
use, offer to sell, sell, import, and otherwise transfer the Work,
where such license applies only to those patent claims licensable
by such Contributor that are necessarily infringed by their
Contribution(s) alone or by combination of their Contribution(s)
with the Work to which such Contribution(s) was submitted. If You
institute patent litigation against any entity (including a
cross-claim or counterclaim in a lawsuit) alleging that the Work
or a Contribution incorporated within the Work constitutes direct
or contributory patent infringement, then any patent licenses
granted to You under this License for that Work shall terminate
as of the date such litigation is filed.
4. Redistribution. You may reproduce and distribute copies of the
Work or Derivative Works thereof in any medium, with or without
modifications, and in Source or Object form, provided that You
meet the following conditions:
(a) You must give any other recipients of the Work or
Derivative Works a copy of this License; and
(b) You must cause any modified files to carry prominent notices
stating that You changed the files; and
(c) You must retain, in the Source form of any Derivative Works
that You distribute, all copyright, patent, trademark, and
attribution notices from the Source form of the Work,
excluding those notices that do not pertain to any part of
the Derivative Works; and
(d) If the Work includes a "NOTICE" text file as part of its
distribution, then any Derivative Works that You distribute must
include a readable copy of the attribution notices contained
within such NOTICE file, excluding those notices that do not
pertain to any part of the Derivative Works, in at least one
of the following places: within a NOTICE text file distributed
as part of the Derivative Works; within the Source form or
documentation, if provided along with the Derivative Works; or,
within a display generated by the Derivative Works, if and
wherever such third-party notices normally appear. The contents
of the NOTICE file are for informational purposes only and
do not modify the License. You may add Your own attribution
notices within Derivative Works that You distribute, alongside
or as an addendum to the NOTICE text from the Work, provided
that such additional attribution notices cannot be construed
as modifying the License.
You may add Your own copyright statement to Your modifications and
may provide additional or different license terms and conditions
for use, reproduction, or distribution of Your modifications, or
for any such Derivative Works as a whole, provided Your use,
reproduction, and distribution of the Work otherwise complies with
the conditions stated in this License.
5. Submission of Contributions. Unless You explicitly state otherwise,
any Contribution intentionally submitted for inclusion in the Work
by You to the Licensor shall be under the terms and conditions of
this License, without any additional terms or conditions.
Notwithstanding the above, nothing herein shall supersede or modify
the terms of any separate license agreement you may have executed
with Licensor regarding such Contributions.
6. Trademarks. This License does not grant permission to use the trade
names, trademarks, service marks, or product names of the Licensor,
except as required for reasonable and customary use in describing the
origin of the Work and reproducing the content of the NOTICE file.
7. Disclaimer of Warranty. Unless required by applicable law or
agreed to in writing, Licensor provides the Work (and each
Contributor provides its Contributions) on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
implied, including, without limitation, any warranties or conditions
of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A
PARTICULAR PURPOSE. You are solely responsible for determining the
appropriateness of using or redistributing the Work and assume any
risks associated with Your exercise of permissions under this License.
8. Limitation of Liability. In no event and under no legal theory,
whether in tort (including negligence), contract, or otherwise,
unless required by applicable law (such as deliberate and grossly
negligent acts) or agreed to in writing, shall any Contributor be
liable to You for damages, including any direct, indirect, special,
incidental, or consequential damages of any character arising as a
result of this License or out of the use or inability to use the
Work (including but not limited to damages for loss of goodwill,
work stoppage, computer failure or malfunction, or any and all
other commercial damages or losses), even if such Contributor
has been advised of the possibility of such damages.
9. Accepting Warranty or Additional Liability. While redistributing
the Work or Derivative Works thereof, You may choose to offer,
and charge a fee for, acceptance of support, warranty, indemnity,
or other liability obligations and/or rights consistent with this
License. However, in accepting such obligations, You may act only
on Your own behalf and on Your sole responsibility, not on behalf
of any other Contributor, and only if You agree to indemnify,
defend, and hold each Contributor harmless for any liability
incurred by, or claims asserted against, such Contributor by reason
of your accepting any such warranty or additional liability.
END OF TERMS AND CONDITIONS

50
README.asciidoc Normal file
View File

@ -0,0 +1,50 @@
:toc:
:toc-placement!:
ifdef::env-github[]
:tip-caption: :bulb:
:note-caption: :information_source:
:important-caption: :heavy_exclamation_mark:
:caution-caption: :fire:
:warning-caption: :warning:
endif::[]
# lxbuildenv - Simple LiteX Build Environment
lxbuildenv is a Python module that takes care of setting up your build environment for LiteX. It is simple to use, cross-platform, and self-contained.
## Usage - Initial Setup
NOTE: The initial setup process is, admittedly, a little rough. Patches are welcome.
You can execute `lxbuildenv.py` directly. Run `python lxbuildenv.py --init` to create a new project. This will perform the following steps:
1. Ask you for a main entrypoint name, if `--main` was not set
1. Create a git repository, if one does not already exist
1. Create a `bin` directory and populate it, if `--no-bin` was not set
1. Create an initial template program
From this point onwards, you do not need to invoke `lxbuildenv.py`. However, it must still be present in order for your program to run.
## Usage - Running your Program
The purpose of `lxbuildenv.py` is to make it easier to get started on a project. It's also designed to help you work on support libraries. It tries to use system dependencies whenever possible, including using system Python and system compilers.
As such, `lxbuildenv.py` does some tricky things with `PYTHONPATH`. It also re-executes itself to ensure `PYTHONHASHSEED` is set. Additionally, it ensures you have any dependencies installed already.
It does not download dependencies for you -- for certain software that's not even legally allowed. Instead it makes sure the software exists and is usable, and where to obtain it if it's not available.
The `lxbuildenv.py` script adds several command line arguments to your program. These all begin with `--lx-`. To list all parameters provided by `lxbuildenv.py`, run `--lx-help`.
WARNING: Because it adjusts the import path, you must `import lxbuildenv` before you import any other packages.
### Specifying Dependencies
Dependencies are specified by adding a variable in your main file called `LX_DEPENDENCIES`. The fact that this is a variable rather than some other method should set off alarm bells and tell you something funny is going on in `lxbuildenv.py`. This is true.
As part of its initialization, `lxbuildenv.py` will read in your program and look for this variable. It will then check any dependencies you've specified to ensure they are met, and may patch up the environment if necessary. For example, if you specify `vivado` as a dependency, `lxbuildenv` will make sure it can run the `vivado` program, and will add `/opt/Xilinx/.../` or `C:\\Xilinx\\...` to the `PATH` as necessary.
To list all possible dependencies, run `python lxbuildenv.py --lx-print-deps`. If you want to try building something without having all the necessary dependencies, add `--lx-ignore-deps` to your build.
## Usage - Tools
As part of its setup, `lxbuildenv.py` creates a `bin/` directory. These are all very thin wrappers around existing tools. You can run these tools using `python`. For example, `python bin/litex_server`.

427
lxbuildenv.py Normal file
View File

@ -0,0 +1,427 @@
#!/usr/bin/env python3
# This script enables easy, cross-platform building without the need
# to install third-party Python modules.
import sys
import os
import subprocess
import argparse
DEPS_DIR = "deps"
# Obtain the path to this script, plus a trailing separator. This will
# be used later on to construct various environment variables for paths
# to a variety of support directories.
script_path = os.path.dirname(os.path.realpath(__file__)) + os.path.sep
# Look through the specified file for known variables to get the dependency list
def get_required_dependencies(filename):
import ast
# Always check the Python version
dependencies = {
'python': 1
}
main_src = ""
try:
with open(sys.argv[0], 'r') as f:
main_src = f.read()
main_ast = ast.parse(main_src, filename=filename)
except:
return list(dependencies.keys())
# Iterate through the top-level nodes looking for variables named
# LX_DEPENDENCIES or LX_DEPENDENCY and get the values that are
# assigned to them.
for node in ast.iter_child_nodes(main_ast):
if isinstance(node, ast.Assign):
value = node.value
for target in node.targets:
if isinstance(target, ast.Name):
if target.id == "LX_DEPENDENCIES" or target.id == "LX_DEPENDENCY":
if isinstance(value, (ast.List, ast.Tuple)):
for elt in value.elts:
if isinstance(elt, ast.Str):
dependencies[elt.s] = 1
elif isinstance(value, ast.Str):
dependencies[value.s] = 1
# Set up sub-dependencies
if 'riscv' in dependencies:
dependencies['make'] = 1
return list(dependencies.keys())
def get_python_path(script_path, args):
# Python has no concept of a local dependency path, such as the C `-I``
# switch, or the nodejs `node_modules` path, or the rust cargo registry.
# Instead, it relies on an environment variable to append to the search
# path.
# Construct this variable by adding each subdirectory under the `deps/`
# directory to the PYTHONPATH environment variable.
python_path = []
if os.path.isdir(script_path + DEPS_DIR):
for dep in os.listdir(script_path + DEPS_DIR):
dep = script_path + DEPS_DIR + os.path.sep + dep
if os.path.isdir(dep):
python_path.append(dep)
return python_path
def fixup_env(script_path, args):
os.environ["PYTHONPATH"] = os.pathsep.join(get_python_path(script_path, 0))
# Set the "LXBUILDENV_REEXEC" variable to prevent the script from continuously
# reinvoking itself.
os.environ["LXBUILDENV_REEXEC"] = "1"
# Python randomizes the order in which it traverses hashes, and Migen uses
# hashes an awful lot when bringing together modules. As such, the order
# in which Migen generates its output Verilog will change with every run,
# and the addresses for various modules will change.
# Make builds deterministic so that the generated Verilog code won't change
# across runs.
os.environ["PYTHONHASHSEED"] = "1"
# Some Makefiles are invoked as part of the build process, and those Makefiles
# occasionally have calls to Python. Ensure those Makefiles use the same
# interpreter that this script is using.
os.environ["PYTHON"] = sys.executable
# Set the environment variable "V" to 1. This causes Makefiles to print
# the commands they run, which makes them easier to debug.
if args.lx_verbose:
os.environ["V"] = "1"
# If the user just wanted to print the environment variables, do that and quit.
if args.lx_print_env:
print("PYTHONPATH={}".format(os.environ["PYTHONPATH"]))
print("PYTHONHASHSEED={}".format(os.environ["PYTHONHASHSEED"]))
print("PYTHON={}".format(sys.executable))
print("LXBUILDENV_REEXEC={}".format(os.environ["LXBUILDENV_REEXEC"]))
sys.exit(0)
# Equivalent to the powershell Get-Command, and kinda like `which`
def get_command(cmd):
if os.name == 'nt':
path_ext = os.environ["PATHEXT"].split(os.pathsep)
else:
path_ext = [""]
for ext in path_ext:
for path in os.environ["PATH"].split(os.pathsep):
if os.path.exists(path + os.path.sep + cmd + ext):
return path + os.path.sep + cmd + ext
return None
def check_python_version(args):
import platform
# Litex / Migen require Python 3.5 or newer. Ensure we're running
# under a compatible version of Python.
if sys.version_info[:3] < (3, 5):
return (False,
"python: You need Python 3.5+ (version {} found)".format(sys.version_info[:3]))
return (True, "python 3.5+: ok (Python {} found)".format(platform.python_version()))
def check_vivado(args):
vivado_path = get_command("vivado")
if vivado_path == None:
# Look for the default Vivado install directory
if os.name == 'nt':
base_dir = r"C:\Xilinx\Vivado"
else:
base_dir = "/opt/Xilinx/Vivado"
if os.path.exists(base_dir):
for file in os.listdir(base_dir):
bin_dir = base_dir + os.path.sep + file + os.path.sep + "bin"
if os.path.exists(bin_dir + os.path.sep + "vivado"):
os.environ["PATH"] += os.pathsep + bin_dir
vivado_path = bin_dir
break
if vivado_path == None:
return (False, "toolchain not found in your PATH", "download it from https://www.xilinx.com/support/download.html")
return (True, "found at {}".format(vivado_path))
def check_cmd(args, cmd, name=None, fix=None):
if name is None:
name = cmd
path = get_command(cmd)
if path == None:
return (False, name + " not found in your PATH", fix)
return (True, "found at {}".format(path))
def check_make(args):
return check_cmd(args, "make", "GNU Make")
def check_riscv(args):
return check_cmd(args, "riscv64-unknown-elf-gcc", "riscv toolchain", "download it from https://www.sifive.com/products/tools/")
def check_yosys(args):
return check_cmd(args, "yosys")
def check_arachne(args):
return check_cmd(args, "arachne-pnr")
dependency_checkers = {
'python': check_python_version,
'vivado': check_vivado,
'make': check_make,
'riscv': check_riscv,
'yosys': check_yosys,
'arachne-pnr': check_arachne,
}
# Validate that the required dependencies (Vivado, compilers, etc.)
# have been installed.
def check_dependencies(args, dependency_list):
dependency_errors = 0
for dependency_name in dependency_list:
if not dependency_name in dependency_checkers:
print('WARNING: Unrecognized dependency "{}"'.format(dependency_name))
continue
result = dependency_checkers[dependency_name](args)
if result[0] == False:
if len(result) > 2:
print('{}: {} -- {}'.format(dependency_name, result[1], result[2]))
else:
print('{}: {}'.format(dependency_name, result[1]))
dependency_errors = dependency_errors + 1
elif args.lx_check_deps or args.lx_verbose:
print('dependency: {}: {}'.format(dependency_name, result[1]))
if dependency_errors > 0:
if args.lx_ignore_deps:
print('{} missing dependencies were found but continuing anyway'.format(dependency_errors))
else:
raise SystemExit(str(dependency_errors) +
" missing dependencies were found")
if args.lx_check_deps:
sys.exit(0)
# Return True if the given tree needs to be initialized
def check_module_recursive(root_path, depth, verbose=False):
if verbose:
print('git-dep: checking if "{}" requires updating...'.format(root_path))
# If the directory isn't a valid git repo, initialization is required
if not os.path.exists(root_path + os.path.sep + '.git'):
return True
# If there are no submodules, no initialization needs to be done
if not os.path.isfile(root_path + os.path.sep + '.gitmodules'):
return False
# Loop through the gitmodules to check all submodules
gitmodules = open(root_path + os.path.sep + '.gitmodules', 'r')
for line in gitmodules:
parts = line.split("=", 2)
if parts[0].strip() == "path":
path = parts[1].strip()
if check_module_recursive(root_path + os.path.sep + path, depth + 1, verbose=verbose):
return True
return False
# Determine whether we need to invoke "git submodules init --recurse"
def check_submodules(script_path, args):
if check_module_recursive(script_path, 0, verbose=args.lx_verbose):
print("Missing submodules -- updating")
subprocess.Popen(["git", "submodule", "update",
"--init", "--recursive"], cwd=script_path).wait()
elif args.lx_verbose:
print("Submodule check: Submodules found")
def main(args):
if args.init:
main_name = os.getcwd().split(os.path.sep)[-1] + '.py'
new_main_name = input('What would you like your main program to be called? [' + main_name + '] ')
if new_main_name is not None and new_main_name != "":
main_name = new_main_name
print("Initializing git repository")
if not os.path.exists(DEPS_DIR):
os.mkdir(DEPS_DIR)
os.system("git init")
os.system("git add " + str(__file__))
os.system("git submodule add https://github.com/m-labs/migen.git deps/migen")
os.system("git add deps/migen")
os.system("git submodule add https://github.com/enjoy-digital/litex.git deps/litex")
os.system("git add deps/litex")
os.system("git submodule add https://github.com/enjoy-digital/litescope deps/litescope")
os.system("git add deps/litescope")
os.system("git submodule add https://github.com/pyserial/pyserial.git deps/pyserial")
os.system("git add deps/pyserial")
os.system("git submodule update --init --recursive")
bin_tools = {
'litex_server': 'litex.soc.tools.remote.litex_server',
'litex_term': 'litex.soc.tools.litex_term',
'mkmscimg': 'litex.soc.tools.mkmscimg',
'mkmscimg': 'litex.soc.tools.mkmscimg',
'litex_sim': 'litex.boards.targets.sim',
'litex_simple': 'litex.boards.targets.simple',
}
bin_template = """
#!/usr/bin/env python3
import sys
import os
# This script lives in the "bin" directory, but uses a helper script in the parent
# directory. Obtain the current path so we can get the absolute parent path.
script_path = os.path.dirname(os.path.realpath(
__file__)) + os.path.sep + os.path.pardir + os.path.sep
sys.path.insert(0, script_path)
import lxbuildenv
"""
# Create binary programs under bin/
if not os.path.exists("bin"):
print("Creating binaries")
os.mkdir("bin")
for bin_name, python_module in bin_tools.items():
with open('bin' + os.path.sep + bin_name, 'w') as new_bin:
new_bin.write(bin_template)
new_bin.write('from ' + python_module + ' import main\n')
new_bin.write('main()\n')
os.system('git add --chmod=+x bin' + os.path.sep + bin_name)
with open(main_name, 'w') as m:
program_template = """#!/usr/bin/env python3
# This variable defines all the external programs that this module
# relies on. lxbuildenv reads this variable in order to ensure
# the build will finish without exiting due to missing third-party
# programs.
LX_DEPENDENCIES = ["riscv", "vivado"]
# Import lxbuildenv to integrate the deps/ directory
import lxbuildenv
# Disable pylint's E1101, which breaks completely on migen
#pylint:disable=E1101
from migen import *
from litex.build.generic_platform import *
_io = [
("clk50", 0, Pins("J19"), IOStandard("LVCMOS33")),
]
class Platform(XilinxPlatform):
def __init__(self, toolchain="vivado", programmer="vivado", part="35"):
part = "xc7a" + part + "t-fgg484-2"
def create_programmer(self):
if self.programmer == "vivado":
return VivadoProgrammer(flash_part="n25q128-3.3v-spi-x1_x2_x4")
else:
raise ValueError("{} programmer is not supported"
.format(self.programmer))
def do_finalize(self, fragment):
XilinxPlatform.do_finalize(self, fragment)
class BaseSoC(SoCSDRAM):
csr_peripherals = [
"ddrphy",
# "dna",
"xadc",
"cpu_or_bridge",
]
csr_map_update(SoCSDRAM.csr_map, csr_peripherals)
def __init__(self, platform, **kwargs):
clk_freq = int(100e6)
def main():
platform = Platform()
soc = BaseSoC(platform)
builder = Builder(soc, output_dir="build", csr_csv="test/csr.csv")
vns = builder.build()
soc.do_exit(vns)
if __name__ == "__main__":
main()
"""
m.write(program_template)
return
# For the main command, parse args and hand it off to main()
if __name__ == "__main__":
parser = argparse.ArgumentParser(
description="Wrap Python code to enable quickstart",
add_help=False)
parser.add_argument(
"-h", "--help", help="show this help message and exit", action="help"
)
parser.add_argument(
'-i', '--init', help='initialize a new project', action="store_true"
)
args = parser.parse_args()
main(args)
elif not os.path.isfile(sys.argv[0]):
print("lxbuildenv doesn't operate while in interactive mode")
elif "LXBUILDENV_REEXEC" not in os.environ:
parser = argparse.ArgumentParser(
description="Wrap Python code to enable quickstart",
add_help=False)
parser.add_argument(
"--lx-verbose", help="increase verboseness of some processes", action="store_true"
)
parser.add_argument(
"--lx-print-env", help="print environment variable listing for pycharm, vscode, or bash", action="store_true"
)
parser.add_argument(
"--lx-check-deps", help="check build environment for dependencies such as compiler and fpga tools and then exit", action="store_true"
)
parser.add_argument(
"--lx-print-deps", help="print all possible dependencies and then exit", action="store_true"
)
parser.add_argument(
"--lx-help", action="help"
)
parser.add_argument(
"--lx-ignore-deps", help="try building even if dependencies are missing", action="store_true"
)
(args, rest) = parser.parse_known_args()
if args.lx_all_deps:
print('Known dependencies:')
for dep in dependency_checkers.keys():
print(' {}'.format(dep))
print('To define a dependency, add a variable inside {} at the top level called LX_DEPENDENCIES and assign it a list or tuple.'.format(sys.argv[0]))
print('For example:')
print('LX_DEPENDENCIES = ("riscv", "vivado")')
sys.exit(0)
deps = get_required_dependencies(sys.argv[0])
fixup_env(script_path, args)
check_dependencies(args, deps)
check_submodules(script_path, args)
try:
sys.exit(subprocess.Popen(
[sys.executable] + [sys.argv[0]] + rest).wait())
except:
sys.exit(1)
else:
# Overwrite the deps directory.
# Because we're running with a predefined PYTHONPATH, you'd think that
# the DEPS_DIR would be first.
# Unfortunately, setuptools causes the sitewide packages to take precedence
# over the PYTHONPATH variable.
# Work around this bug by inserting paths into the first index.
for path in get_python_path(script_path, None):
sys.path.insert(0, path)