Python the Language (in case of)
Related:
- learning Django
- python desktop GUI/frontend
- py code cookbook/reuse
- plot in py w pyvis
- Python in Win by MS official tutorial CN
- 用 Rust 让 numpy、scikit 和 pandas 加速 100 倍!开源 Weld 技术揭秘
INSTALL PY3
win
Install py2 first! Install py3 later.
Download and install py3 from the official website.
During installation, make sure to "add to system PATH".
If it is only in user's "Path", we need to also add to system "PATH" (both the python root folder and the script subfolder), and make sure it is before py2's folders.
ubuntu
Plz use pyenv below.
(Ubuntu 16.04 has both py2 & py3; default py2.)
INSTALL PIP IN UBUNTU 16
sudo apt-get update && sudo apt-get -y upgrade && \
sudo apt-get install python-pip python3-pip && \
pip -V; pip3 -V
# use a local index/source instead of https://pypi.python.org/simple/
pip install -i https://pypi.tuna.tsinghua.edu.cn/simple/ pip -U
pip config set global.index-url https://pypi.tuna.tsinghua.edu.cn/simple/
Alternative indexes/sources[ref]:
- 清华大学 https://pypi.tuna.tsinghua.edu.cn/simple/
- 阿里云 http://mirrors.aliyun.com/pypi/simple/
- 中国科技大学 https://pypi.mirrors.ustc.edu.cn/simple/
- 豆瓣(douban) http://pypi.douban.com/simple/
- 中国科学技术大学 http://pypi.mirrors.ustc.edu.cn/simple/ (302 redirected to mirrors.tuna.tsinghua.edu.cn)
FREQUENT PIP CMD
pip install package_name
pip uninstall package_name
pip search package_name
pip freeze
pip --help
JUPYTER
see: jupyter
PYENV
install
Before installing any specific py version: sudo apt install -y python3-dev libbz2-dev libssl-dev libsqlite3-dev libreadline-dev lib32readline-dev libffi-dev build-essential zlib1g-dev
. (see the common error below for the reason)
Run curl[ref]:
(manually do it if lacking curl)
Note: every user needs to run individually:
curl https://pyenv.run | bash
Update .bashrc:
# pyenv
if [ -d "$HOME/.pyenv" ]; then
export PYENV_ROOT="$HOME/.pyenv"
export PATH="$PYENV_ROOT/bin:$PATH"
eval "$(pyenv init -)"
eval "$(pyenv virtualenv-init -)"
fi
Then:
source ~/.bash_profile
Install python:
pyenv install 2.7.8
common error (Ubuntu20)
QA: a common error when installing a python version using pyenv install ...
:
WARNING: The Python bz2 extension was not compiled. Missing the bzip2 lib
WARNING: The Python readline extension was not compiled. Missing the GNU readline lib
ERROR: The Python ssl extension was not compiled. Missing the OpenSSL lib
Solution:
See above "Before installing any specific py version". Note: may need to search by: apt-cache search readline |grep dev
[ref libffi requies re-install of py3]
check py version
Tip: to find py version:
# in terminal:
python --version
# in py/jupyter:
import sys
sys.version_info
print(sys.executable) # https://stackoverflow.com/a/46902466
print(sys.version)
print(sys.version_info)
PIPENV
A state-of-the-art combined tool chain.
install
Tip: better to have pyenv installed also, so any python version can be automatically installed by pipenv.
sudo apt install -y python3-pip python3-setuptools && \
python3 -m pip install pipenv # Note: plz use --user when installing pipenv for non-root users using sudo
cat >>~/.bash_profile <<EOD
export PATH="\$HOME/.local/bin:\$PATH"
EOD
source ~/.bash_profile
create env
mkdir sunnyProj; cd sunnyProj
pipenv --python 3.6.7 # will automatically install the specified python in env, if pyenv is installed.
(note: if got error, try manually install a python version by pyenv install x.x.x
w/o 'rc', then pipenv ...
again.)
activate env
pipenv shell
how it works
A pip-environment is based on one folder.
Actually, the Pipfile
in the folder.
When creating a new environment, it will use the folder's name as the environment's name, and will also search for the environment using the folder's anme when trying to activating by pipenv shell
.
When activating one environment, a specific Python's installation path and Python lib path will be added to PATH and have higher priority than system one.
"Pipenv" uses "virtualenvs" as the backend (?).
Ref:
Install and kick-start, (originally from phoikoi.io)
pipenv (bak) (we can use pipenv --python
directly as it will install the related python)
list of packages/libraries
Bash:
pip freeze
# or:
pip list
Python:
import pip
installed_packages = pip.get_installed_distributions()
installed_packages_list = sorted(["%s==%s" % (i.key, i.version)
for i in installed_packages])
print(installed_packages_list)
# OR (one-line)
sorted(["%s==%s" % (i.key, i.version) for i in pip.get_installed_distributions()])
Ref: stackoverflow
pipenv + jupyter kernels
see /jupyter
VIRTUAL ENV
There are 2 tools (virtualenv
and virtualenvwrapper
) can be used. There is also one blog introducing bash alias way to use virtualenv
: ref ( evernote backup ).
We will use the wrapper one (cuz we prefer new stuff, right?).
If we need to put the env into a folder other than the user's home, we can use the old virtualenv
package:
Win: pip install virtualenvwrapper-win
Nix: pip install virtualenvwrapper
(this also works for windows with cygwin)
If we want to use another python intepretor, we can use the optional -p
(path) parameter:
TIP: After running this command, we will automatically enter the virtual environment.
The name is showing at the beginning of terminal/cmd prompt, surrended by ( )
:
OBS: newly created env does _NOT_ have globally installed packages.
When we are in an env, we can use pip
to install packages for this env.
Take Django as an example:
python
import django
print(django.get_version())
Even with -p
parameter, we need to run the .py files with virtual env's python.exe, e.g.:
Otherwise, system's global intepretor will take effect.
We can use D:\Python27>mklink python27.exe python.exe
to make it possible to run server by: python27.exe manage.py runserver
. However, direct copying python.exe and renaming the file does NOT work.
POETRY (alt. pipenv)
install
curl -sSL https://raw.githubusercontent.com/python-poetry/poetry/master/get-poetry.py | python -
poetry config virtualenvs.in-project true
add to project config file pyproject.toml
:
[[tool.poetry.source]]
name = "pypi-tuna-simple"
url = "https://pypi.tuna.tsinghua.edu.cn/simple/"
default = true
usage
pyenv shell 3.6.9
poetry shell
note
- advantages: good dependency resolver among libraries; also support jupyter notebook/lab.
- disadvantages: poor dependency resolver when python version does not match (need to manually specify ); cannot automatically call pyenv, manual work needed.
PARALLEL
PERFORMANCE
Simple seconds: time()
import time
start_time = time.time()
# do something here
end_time = time.time()
print(end_time - start_time, 'seconds.')
Alternatives: perf_counter
or process_time
ref stackoverflow.
Python in Browser
Use Py instead of js (javascript) by pyodide.
Try directly here.
Maybe also Juliea & R later.
MORE
HANDY LIBS
python -m http.server [8000] # default port 8000, seems no need to install
python -m SimpleHTTPServer [8000] # default port 8000
PACK PY PROJECTS (django) AS .EXE FOR WINDOWS
winpython
pyinstaller tips
- Check the supported python version and django versions even before coding.
-
(a good alt. is the .org official one or the winpython dist.)
Problem: "RuntimeError: Script 'runserver' does not exist"
Reason: // possible compatibility issue due to newer django
Solution: manage runserver --noreload
[ref]
Problem: ModuleNotFoundError ...
Reason 1: using custom app verbose_name in my_site/my_app/apps.py
Solution 1: not use :(
Solution 2: see the csdn ref below, use --hidden-import
and/or hook during compiling (more problems: need to import xxx.urls etc.)
Solution 3: import them in manage.py before compiling instead of using --hidden-import
etc. flags.
Reason: other.
Solution: csdn
Problem: ImportError: cannot import name _elementpath
Reason: using python-docx (and some deeper reasons)
Solution: in manage.py: from lxml import _elementpath
Problem: template not found.
Reason: pyinstaller cannot identify & copy html templates & static folders.
Solution: manual copy. (a better one???)
Problem: .pyc is not safe.
Reason: every single line can be de-compiled easily from .pyc files, including comments.
Solution: to .pyd by from Cython.Build import cythonize ...
[ref cn]
Problem: "...ImproperlyConfigured: Requested setting DJANGO_TABLES2_TEMPLATE, but settings are not configured."
Reason: lacks the config.
Solution: DJANGO_TABLES2_TEMPLATE = "django_tables2/bootstrap4.html"
in settings.py
. [refs: doc 1.11.2 & blog] and SET DJANGO_SETTINGS_MODULE=<my_site_name>.settings
(like export in nix) before running [ref1, [ref2]].
Problem: static file dir config not working when DEBUG=False.
Reason: ...
Solution: ...
Problem: pyinstaller compiled exe files detected as virus.
Reason & solution: https://stackoverflow.com/a/52054580
Solution NOT working !
For meanings of <my_site_name>.spec
, see ref cn (evernote bak)
some more tips: [full process in linux, csdn 2020]; [hidden imports, csdn 2020]
some more pitfalls (mentioning similar ones but different solutions):