=encoding utf-8 =head1 Name opm - OpenResty Package Manager =head1 Status Experimental. =head1 Synopsis For library users: # show usage opm --help # search package names and abstracts with the user pattern "lock". opm search lock # search package names and abstracts with multiple patterns "lru" and "cache". opm search lru cache # install a package named lua-resty-foo under the name of some_author opm get some_author/lua-resty-foo # get a list of lua-resty-foo packages under all authors. opm get lua-resty-foo # show the details of the installed package specified by name. opm info lua-resty-foo # show all the installed packages. opm list # upgrade package lua-resty-foo to the latest version. opm upgrade lua-resty-foo # update all the installed packages to their latest version. opm update # uninstall the newly installed package opm remove lua-resty-foo All the commands can follow the C<--cwd> option to work in the current working directory (under ./resty/modules/) instead of the system-wide location. # install into ./resty_modules/ instead of the system-wide location: opm --cwd get foo/lua-resty-bar # check the locally installed packages under ./resty_modules/ opm --cwd list # remove the locally installed packages under ./resty_modules/ opm --cwd remove lua-resty-bar For library authors: cd /path/to/lua-resty-foo/ opm build # optional: # cd lua-resty-foo-VERSION/ && opm server-build # you may need to edit the ~/.opmrc file to set up your github # personal access tokens. the first run of "opm upload" will create # a boilerplate ~/.opmrc file for you. opm upload # cleaning up the leftovers of the opm build command. opm clean dist =head1 Description C is the official OpenResty package manager, similar to Perl's CPAN and NodeJS's npm in rationale. We provide both the C client-side command-line utility and the server-side application for the central package repository in this GitHub code repository. The C command-line utility can be used by OpenResty users to download packages published on the central C server (i.e., C). It can also be used to package and upload the OpenResty package to the server for package authors and maintainers. You can find the source of C under the C directory. It is currently implemented as a standalone Perl script. The server side web application is built upon OpenResty and written in Lua. You can find the server code under the C directory. Unlike many other package management systems like C, C, C, or C. Our C adopts a package naming discipline similar to C, that is, every package name should be qualified by a publisher ID, as in C where C is the publisher ID while C is the package name itself. This naming requirement voids the temptation of occupying good package names and also allows multiple same-name libraries to coexist in the same central server repository. It is up to the user to decide which library to install (or even install multiple forks of the same library in different projects of hers). For simplicity, we simply map the GitHub user IDs and organization IDs to the publisher IDs for C. For this reason, we use the GitHub personal access tokens (or oauth tokens) to authenticate our package publishers. This also eliminates the sign-up process for C package authors altogether. C has built-in support for the C tool, that is, the documentation of the packages installed via C is already indexed by C and can be viewed directly on the terminal with the C tool. C currently only supports pure Lua libraries but we will add support for Lua libraries in pure C or with some C components very soon. The vision is to also add support for redistributing 3rd-party NGINX C modules as dynamic NGINX modules via C in the future. The OpenResty world consists of various different kinds of "modules" after all. We also have plans to allow the user to install LuaRocks packages via C through the special user ID C. It poses a risk of installing an OpenResty-agnostic Lua module which blocks the NGINX worker processes horribly on network I/O, nevertheless, as the developers of C, we always like choices, especially those given to our users. =head1 Usage opm [options] command package... Options: -h --help Print this help. --install-dir=PATH Install into the specified PATH directory instead of the system-wide OpenResty installation tree containing this tool. --cwd Install into the current working directory under ./resty_modules/ instead of the system-wide OpenResty installation tree containing this tool. Commands: build Build from the current working directory a package tarball ready for uploading to the server. clean ARGUMENT... Do clean-up work. Currently the valid argument is "dist", which cleans up the temporary files and directories created by the "build" command. info PACKAGE... Output the detailed information (or meta data) about the specified packages. Short package names like "lua-resty-lock" are acceptable. get PACKAGE... Fetch and install the specified packages. Fully qualified package names like "openresty/lua-resty-lock" are required. One can also specify a version constraint like "=0.05" and ">=0.01". list List all the installed packages. Both the package names and versions are displayed. remove PACKAGE... Remove (or uninstall) the specified packages. Short package names like "lua-resty-lock" are acceptable. search QUERY... Search on the server for packages matching the user queries in their names or abstracts. Multiple queries can be specified and they must fulfilled at the same time. server-build Build a final package tarball ready for distribution on the server. This command is usually used by the server to verify the uploaded package tarball. update Update all the installed packages to their latest version from the server. upgrade PACKAGE... Upgrade the packages specified by names to the latest version from the server. Short package names like "lua-resty-lock" are acceptable. upload Upload the package tarball to the server. This command always invokes the build command automatically right before uploading. =head2 Global Installation To globally install opm packages, just use the C command. =head2 Local Installation When you use C<--cwd> option to install packages to the C<./resty_modules/> directory, then you should put the following lines to your C, inside the C block: lua_package_path "$prefix/resty_modules/lualib/?.lua;;"; lua_package_cpath "$prefix/resty_modules/lualib/?.so;;"; Do NOT change C<$prefix> to a hard-coded absolute path yourself! OpenResty will automatically resolve the special C<$prefix> variable in the directive values at startup. The C<$prefix> value will be resolved to the server prefix, which will later be specified via the C<-p> option of the C command line. And then you should start your OpenResty application from the current working directory like this: openresty -p $PWD/ assuming you have the following OpenResty application directory layout in the current directory: logs/ conf/ conf/nginx.conf resty_modules/ Alternatively, if you just want to use the C command line utility with the opm modules installed into the C<./resty_modules> directory, then you should just use the C<-I ./resty_modules/lualib> option, as in resty -I ./resty_modules/lualib -e 'require "foo.bar".go()' =head1 HTTP Proxy Support HTTP proxies are supported via the C and C system environment variables, as in http_proxy [protocol://][:port] Sets the proxy server to use for HTTP. https_proxy [protocol://][:port] Sets the proxy server to use for HTTPS. =head1 Author Workflow The package author should put a meta-data file named C on the top-level of the Lua library source tree. This file is used by the C command to build and package up your library into a tarball file which can be later uploaded to the central package server via the C command. One example C file looks like below for OpenResty's L library: # distribution config for opm packaging name = lua-resty-core abstract = New FFI-based Lua API for the ngx_lua module author = Yichun "agentzh" Zhang (agentzh) is_original = yes license = 2bsd lib_dir = lib doc_dir = lib repo_link = https://github.com/openresty/lua-resty-core main_module = lib/resty/core/base.lua requires = luajit, openresty/lua-resty-lrucache >= 0.04 As we can see, the C file is using the popular L. Most of the fields in this example should be self-explanatory. For detailed documentation for the fields available in C, please check out the L section. The C command also reads and extracts information from the configuration file C<.opmrc> under the current system user's home directory (i.e., with the file path C<~/.opmrc>). If the file does not exist, C will automatically generates a boilerplate file in that path. One sample C<~/.opmrc> file looks like this. # your github account name (either your github user name or github organization that you owns) github_account=agentzh # you can generate a github personal access token from the web UI: https://github.com/settings/tokens # IMPORTANT! you are required to assign the scopes "user:email" and "read:org" to your github token. # you should NOT assign any other scopes to your token due to security considerations. github_token=0123456789abcdef0123456789abcdef01234567 # the opm central servers for uploading openresty packages. upload_server=https://opm.openresty.org download_server=https://opm.openresty.org Basically, the C command just needs the C setting from this file. Other fields are needed by the C command that tries to upload the packaged tarball onto the remote package server. You can either use your own GitHub login name (which is C in this example), or a GitHub organization name that you owns (i.e., having the C permission to it). After C successfully generates a C<.tar.gz> file under the current working directory, the author can use the C command to upload that file to the remote server. To ensure consistency, C automatically runs C itself right before attempting the uploading operation. The central package server (C in this case) calls the GitHub API behind the scene to validate the author's identify. Thus the author needs to provide his GitHub personal access token in her C<~/.opmrc> file. Only the C and C permissions (or C in the GitHub terms) need to be assigned to this access token. =head1 File dist.ini The C file specifies the meta data of a package and is used by C to generate a tarball that is ready to upload to the remote package server. This file should sit at the top of the library or module source tree. This file uses the L. It contains the following keys (or properties) in the default top-level section: =head2 name Specifies the name of the package (excluding version numbers). For example, name = lua-resty-limit-traffic The name can only contain letters, digits, and dashes (C<->). This key is mandatory. =head2 abstract Abstract for the current package. abstract = New FFI-based Lua API for the ngx_lua module You can use UTF-8 characters in this field value. Invalid UTF-8 sequences, however, will lead to errors in C or C commands. This key is mandatory. =head2 version Version number for the current package. If this key is specified, then the version number specified here will be automatically compared with the version number extracted from the "main module" file (see the L key for more details). Example: version = 1.0.2 See also the L section for more details on package version numbers. This key is optional. =head2 author Specifies one or more authors of the libraries. For instance, author = Yichun Zhang (agentzh) The names of multiple authors should be separated by a comma, with optional surrounding spaces. author = Yichun Zhang (agentzh), Dejiang Zhu You can use UTF-8 characters in this field value. Invalid UTF-8 sequences, however, will lead to errors in C or C commands. This key is mandatory. =head2 license Specifies the license for the library. For example, license = 3bsd This assigns the 3-clause BSD license to the current package. Special IDs for common code licenses are required. For now, the following IDs are supported: =over =item * C<2bsd> =back BSD 2-Clause "Simplified" or "FreeBSD" license =over =item * C<3bsd> =back BSD 3-Clause "New" or "Revised" license =over =item * C =back Apache License 2.0 =over =item * C =back Artistic License =over =item * C =back Artistic License 2.0 =over =item * C =back Creative Commons Attribution 4.0 International Public License =over =item * C =back Creative Commons Attribution-ShareAlike 4.0 International Public License =over =item * C =back Creative Commons Attribution-NoDerivatives 4.0 International Public License =over =item * C =back Creative Commons Attribution-NonCommercial 4.0 International Public License =over =item * C =back Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International Public License =over =item * C =back Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International Public License =over =item * C =back Common Development and Distribution License =over =item * C =back Eclipse Public License =over =item * C =back GNU General Public License (GPL) =over =item * C =back GNU General Public License (GPL) version 2 =over =item * C =back GNU General Public License (GPL) version 3 =over =item * C =back GNU Library or "Lesser" General Public License (LGPL) =over =item * C =back MIT license =over =item * C =back Mozilla Public License 2.0 =over =item * C =back Proprietary =over =item * C =back Public Domain If you do need an open source license not listed above, please let us know. It is also possible to specify multiple licenses at the same time, as in license = gpl2, artistic2 This specifies dual licenses: GPLv2 and Artistic 2.0. To upload the package to the official opm package server, you must at least specify an open source license here. This key is mandatory. =head2 requires Specifies the runtime dependencies of this package. Multiple dependencies are separated by commas, with optional surrounding spaces. As in requires = foo/lua-resty-bar, baz/lua-resty-blah All the dependency constraints specified in this key must be met at the same time when the C or C command is run. You can also specify version number requirements, as in requires = foo/lua-resty-bar >= 0.3.5 The version comparison operators supported are C<< >= >>, C<=>, and C<< > >>. Their semantics is self-explanatory. You can also specify the following special dependency names: =over =item * C =back Requires the LuaJIT component in the package user's OpenResty installation (and also the package uploader's). When version number constraints are specified, the version number of the LuaJIT will also be checked. =over =item * C =back Requires the NGINX component in the package user's OpenResty installation (and also the package uploader's). When version number constraints are specified, the version number of the NGINX core will also be checked. =over =item * C =back This dependency only makes sense when there is an associated version number constraint specified. The version number of the package user's (and also uploader's) OpenResty installation must meet the version constraint here. =over =item * C =back Requires the ngx_http_lua_module component in the package user's OpenResty installation (and also the package uploader's). When version number constraints are specified, the version of the installed ngx_http_lua_module will also be checked. Below is such an example: requires = luajit >= 2.1.0, nginx >= 1.11.2, ngx_http_lua = 0.10.6 or you can just specify a single C version constraint to cover them all in the example above: requires = openresty >= 1.11.2.1 This key is optional. =head2 repo_link The URL of the code repository (usually on GitHub). For example, repo_link = https://github.com/openresty/lua-resty-core If the repository is on GitHub, then C ensures that the name specified in the C in your C<~/.opmrc> file I match the account in your GitHub repository URL. Otherwise C reports an error. This key is mandatory. =head2 is_original Takes the value C or C to specify whether this package is an original work (that is, not a fork of another package of somebody else). This key is mandatory. =head2 lib_dir Specifies the root directory of the library files (C<.lua> files, for example). You must not use absolute directory paths or paths containing C<..> as the value. Default to C. This key is optional. =head2 exclude_files Specifies patterns for files to be excluded during packaging via C. Unix shell wildcards like C<*> and C are supported. Multiple patterns should be separated by commas, with optional surrounding spaces. exclude_files=foo*.lua, bar/baz/*/*.lua, lint_config.lua =head2 main_module This key specifies the PATH of the "main module" file of the current package. The C command reads the "main module" file to extract the version number of the current package, for example. C uses simple regular expressions to find Lua code patterns like below: _VERSION = '1.0.2' version = "0.5" version = 0.08 When this key is not specified, then C will try to find the main module file automatically (which might be wrong though). You must not use absolute file paths or paths containing C<..> as the value. This key is optional. =head2 doc_dir Specifies the root directory of the documentation files. Default to C. You must not use absolute directory paths or paths containing C<..> as the value. C always tries to collect the documentation files in either the Markdown (C<.md> or C<.markdown>) or the POD (C<.pod>) format. Regardless of the value of this C key, C always tries to collect the following files in the current working directory (which should be the root of the current package): =over =item * C, C, or C =item * C =item * C =item * C, C, or C =back You can use UTF-8 characters in these documentation files. Other multi-byte character encodings must be avoided. This key is optional. =head1 File .opmrc The C<.opmrc> file under the current system user's home directory configures various important settings for the current system user. Only library authors should care about this file since commands like C, C, or C do not need this file at all. Like L, this file is also in the L. When this file is absent, the first run of the C or C commands will automatically generate a boilerplate file for you to fill out later yourself. This file recognizes the following keys: =head2 github_account Specifies your GitHub account name, either your GitHub user login name or github organization that you owns. For example, the document writer's GitHub login name is C while he also owns the GitHub organization C. So he can choose to upload his packages either under the C or C with the same GitHub access token (defined via the L key) by configuring this C key. This key is required. =head2 github_token Specifies your GitHub personal access token used for package uploads. You can generate a GitHub personal access token from the GitHub L. While you are generating your token on GitHub's web site, it is crucial to assign the right permissions (or C in GitHub's terminology) to your token. The C tool chain requires that the token must contain the C scope. Optionally, you can also assign the C scope at the same time, which is required if you want to upload your OpenResty packages under an organization name that you owns. The GitHub personal access tokens are like passwords, so be very careful when handling it. Never share it with the rest of the world otherwise anybody can upload packages to the OPM package server under I name. Due to security considerations, the package server also rejects GitHub personal access tokens that are too permissive (that is, having more scopes than needed). The package server caches a sorted hash of your tokens in its own database, so that the server does not have to query GitHub upon subsequent uploads. Because the tokens are hashed, the package server can only verifies that your token is correct but cannot recover your original token just from the database. This key is required. =head2 upload_server Specifies the OPM server for uploading packages. Defaults to C. It is strongly recommended to use C (which is the default) for communication privacy. The official OPM package server is C. You could, however, point this key to your own or any 3rd-party servers (then you are at your own risk). This key can have a different value than L. =head2 download_server Specifies the OPM server for downloading packages. Defaults to C. It is strongly recommended to use C (which is the default) for communication privacy. The official OPM package server is C. You could, however, point this key to your own or any 3rd-party servers (then you are at your own risk). This key can have a different value than L. =head1 Version Number Handling OPM requires all package version numbers to only consist of digits, dots, alphabetic letters, and underscores. Only the digits part are mandatory. OPM treats all version numbers as one or more integers separated by dots (C<.>) or any other non-digit characters. Version number comparisons are performed by comparing each integer part in the order of their appearance. For example, the following version number comparisons hold true: 12 > 10 1.0.3 > 1.0.2 1.1.0 > 1.0.9 0.10.0 > 0.9.2 There can be some surprises when your version numbers look like decimal numbers, as in 0.1 < 0.02 This is because C<0.1> is parsed as the integer pair C<{0, 1}>, while C<0.02> is parsed as C<{0, 2}>, so the latter is greater than the former. To avoid such pitfalls, always specify the decimal part of the equal length, that is, writing C<0.1> as C<0.10>, which is of the same length as C<0.02>. OPM does not support special releases like "release candidates" (RC) or "developer releases" yet. But we may add such support in the future. For forward-compatibility, the package author should avoid version numbers with suffixes like C<_2> or C. =head1 Installation =head2 For opm L since C<1.11.2.2> already include and install C by default. So usually you do I need to install C yourself. It worth noting that if you are using the official OpenResty L, you should install the L package since the L binary package itself does not contain C. If you really want to update to the latest version of C in the code repository, then just copy the file C in the repository over to C<< /bin/ >> where C<< >> is the value of the C<--prefix> option of C<./configure> while you are building your OpenResty (defaults to C). # defaults to `/usr/local/openresty/` # unless you override it when building OpenResty yourself. sudo cp bin/opm /bin/ If you are using an older version of OpenResty that does I include C by default, then you should also create the following directories: cd sudo mkdir -p site/lualib site/manifest site/pod Note that at least OpenResty 1.11.2.1 is needed for C to work properly. To run the C tool, you just need C, C, and C to run the C tool. Ensure that your perl is not too old (should be at least C<5.10.1>), and your curl supports C. =head1 Security Considerations The C client tool always uses HTTPS to talk to the package server, L, by default. Both for package uploading and package downloading, as well as other web service queries for meta data. Although it is possible for the user to manually switch to the HTTP protocol by editing the C and/or C keys in her own C<~/.opmrc> file. The C client tool also always verifies the SSL certificates of the remote OPM package server (via C right now). Similarly, the OPM package server always uses TLS to talk to remote services provided by GitHub and Mailgun. These remote sites' SSL certificates are also always verified on the server side. This cannot be turned off by the user. The OPM package server uses PostgreSQL's C extension to encrypt the authors' GitHub personal access tokens in the database (we cache the tokens in our own database to speed up subsequent uploads and improve site reliability when the GitHub API is down). Even the server administrators cannot recover the original access tokens from the database. The server also ensures that the author's personal token is not too permissive by rejecting such tokens. The C tool chain and server also always perform the MD5 checksum verification upon both the downloaded and uploaded package files, to ensure data integrity when transferred over the wire. =head1 Credit The design of the C tool gets various inspirations from various existing package management systems, including but not limited to, Perl's C and L, RedHat's C, NodeJS's C, and Mac OS X's C. =head1 TODO =over =item * Add C command to reinstall an already installed module (at the same version). =item * Add C command to check if there is any inconsistency in the current opm package installation tree. =item * Add C<< opm files >> command to list all the files in the specified package. =item * Add C<< opm whatprovides >> command to find out which package the specified file belongs to. =item * Add plugin mechanisms to C (similar to Perl's L packaging framework). =item * Turn opm.openresty.org into a full-blown web site similar to search.cpan.org. =item * Add support for Lua C modules and LuaJIT FFI modules with standalone C libraries. =item * Add support for 3rd-party NGINX C modules (which can be compiled as NGINX dynamic modules). =item * Add (limited) support for LuaRocks via the special name space C, for example, =back opm get luarocks/foo =head1 Author Yichun Zhang (agentzh) Eagentzh@gmail.comE, OpenResty Inc. =head1 Copyright and License This module is licensed under the BSD license. Copyright (C) 2016-2020, by Yichun "agentzh" Zhang (章亦春) Eagentzh@gmail.comE, OpenResty Inc. All rights reserved. Redistribution and use in source and binary forms, with or without modification, are permitted provided that the following conditions are met: =over =item * Redistributions of source code must retain the above copyright notice, this list of conditions and the following disclaimer. =back =over =item * Redistributions in binary form must reproduce the above copyright notice, this list of conditions and the following disclaimer in the documentation and/or other materials provided with the distribution. =back THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.