Python the Language (in case of)

Python the Language (in case of)

2016-08-21. Category & Tags: Python, Environment, Virtual Environment, UV, Py, Py3, Lang, Language

Related:

PRE-INSTALL: UV, CONDA VS. PIP #

Pip requires apt. Thus, pip+apt. Pip chose to live together with apt (or other package manager) as apt is better at managing packages, especially when related to system. Some pip packages requires system (apt) to cooperate, such as uwsgi, graphviz, ffmpeg, gcc, python3-dev.

For data / AI engineering, uv > conda > pip. Conda has a good system and community to maintain AI/ML related packages, but for tasks beyond AI/ML, pip is sometimes required. It even breaks during env migration and has to be fixed by pip. Thus, conda+pip+apt, which is more complex than pip+apt for engineering other than AI/ML. UV is used to replace Conda as a free solution for business.

ref: zhihu (bak)

VIRTUAL ENV w/ UV #

old/deprecated: /conda-venv: Conda/Anaconda, poetry, pyenv, pipenv etc.

TIP: see /jupyter for virtual env + jupyter kernels.

install #

on win (tested on win 11) # 

powershell -c "irm https://astral.sh/uv/install.ps1 | iex"
$env:Path = "C:\Users\sunbi\.local\bin;$env:Path"

# or in cmd:
# set Path=C:\Users\sunbi\.local\bin;%Path%

nix/mac #

curl -LsSf https://astral.sh/uv/install.sh | sh

pip install uv

to verify #

uv --version

if needed (not yet), init/create a project folder # 

uv init new-proj-name # create a folder with: .git .gitignore .python-version pyproject.toml README.md main.py

# or with py version:
uv init new-proj-name --python 3.11 # influences .python-version

create a venv in an existing project folder # 

uv venv # which creates a ".venv" directory/folder in the current(project) folder

# or with py version:
uv venv --python 3.11 # suggested if having NO .python-version

# 指定包的安装目录（不放在当前目录）
uv venv /home/my_username/uv_env_for_project_1

activate # 

## nix/mac
source .venv/bin/activate

## win
.venv\Scripts\activate

install packages #

WARN: do NOT use pip, which will not mange “pyproject.toml” / “uv.lock”.

ref: https://docs.astral.sh/uv/pip/packages/

a specific package #

uv pip install the_package_name (uv pip uninstall ...)

or: a package with optional dependencies enabled, e.g., Flask with the “dotenv” extra: uv pip install "flask[dotenv]"

or upgrade a package (this also modifies project.toml & uv.lock) uv add --upgrade the_package_name

or downgrade a package: uv add the_package_name==0.28.1

from uv.lock #

uv sync which installs the required packages according to “.python-version” & “uv.lock”.

(migrate) from old requirements.txt #

uv add -r requirements.txt

“add” will add the package to pyproject.toml, and install the package.

multiple python versions # 

uv python install 3.10
uv python use 3.10

tips #

uv run includes uv lock & uv sync before running the actual command, e.g. it .venv folder will be created and packages will be installed if not.

pyproject.toml: TOML format, the direct dependencies of the project, versions like: “eg_package_name>=2.32.3”.

uv.lock: TOML format, the exact versions of dependencies installed (DO NOT manullay modify).

INSTALL PY3 #

win #

Install py2 first! Install py3 later.
Download and install py3 from the official website.
During installation, make sure to “add to system PATH”.
If it is only in user’s “Path”, we need to also add to system “PATH” (both the python root folder and the script subfolder), and make sure it is before py2’s folders.

ubuntu #

Plz use uv venv/pyenv. (Ubuntu 16.04 has both py2 & py3 but default to py2.)

INSTALL PIP IN UBUNTU 16 【dated OS】 #

Suggested: use uv venv.

sudo apt update && sudo apt -y upgrade && \
sudo apt install python-pip python3-pip && \
pip -V; pip3 -V

# use a local index/source instead of https://pypi.python.org/simple/
pip3 install -i https://pypi.tuna.tsinghua.edu.cn/simple/ pip -U
pip3 config set global.index-url https://pypi.tuna.tsinghua.edu.cn/simple/

Alternative indexes/sources [ref]:

FREQUENT PIP CMD # 

pip install --prefer-binary package_name
pip uninstall package_name
pip search package_name
pip freeze # only the ones installed by `pip install xxx`
pip freeze > requirements.txt
pip freeze --all > requirements.txt
pip --help

PARALLEL #

PERFORMANCE #

Simple seconds: time()

import time
start_time = time.time()
# do something here
end_time = time.time()
print(end_time - start_time, 'seconds.')

Alternatives: perf_counter or process_time
ref stackoverflow.

Python in Browser #

Use Py instead of js (javascript) by pyodide.
Try directly here.
Maybe also Juliea & R later.

MORE #

automated lazy-loading of common libs: pip install --prefer-binary pyforest
不可不知的 Python 陷阱
Kite: Python 代码补全利器

HANDY LIBS # 

python -m http.server [8000] # default port 8000, build-in, no need to install
python -m SimpleHTTPServer  [8000] # default port 8000

PACK PY PROJECTS (django) AS .EXE FOR WINDOWS #

winpython #

pyinstaller tips #

WARNING !!!

  • Check the supported python version and django versions even before coding.
  • The Python distribution version from windows store is lacking something, so DO NOT USE !!!
    (a good alt. is the .org official one or the winpython dist.)

Problem: “RuntimeError: Script ‘runserver’ does not exist”
Reason: // possible compatibility issue due to newer django
Solution: manage runserver --noreload [ref]

Problem: ModuleNotFoundError ...
Reason 1: using custom app verbose_name in my_site/my_app/apps.py
Solution 1: not use :(
Solution 2: see the csdn ref below, use --hidden-import and/or hook during compiling (more problems: need to import xxx.urls etc.)
Solution 3: import them in manage.py before compiling instead of using --hidden-import etc. flags.
Reason: other.
Solution: csdn

Problem: ImportError: cannot import name _elementpath
Reason: using python-docx (and some deeper reasons)
Solution: in manage.py: from lxml import _elementpath

Problem: template not found.
Reason: pyinstaller cannot identify & copy html templates & static folders.
Solution: manual copy. (a better one???)

Problem: .pyc is not safe.
Reason: every single line can be de-compiled easily from .pyc files, including comments.
Solution: to .pyd by from Cython.Build import cythonize ...

Problem: wanna get .py from pyinstaller generated .exe.
Solution: python pyinstxtractor.py manage.exe; pip install --prefer-binary uncompyle6; uncompyle6 model.pyc ref cn

Problem: “…ImproperlyConfigured: Requested setting DJANGO_TABLES2_TEMPLATE, but settings are not configured.”
Reason: lacks the config.
Solution: DJANGO_TABLES2_TEMPLATE = "django_tables2/bootstrap4.html" in settings.py. [refs: doc 1.11.2 & blog] and SET DJANGO_SETTINGS_MODULE=<my_site_name>.settings (like export in nix) before running [ref1, [ref2]].

Problem: static file dir config not working when DEBUG=False.
Reason: …
Solution: …

Problem: pyinstaller compiled exe files detected as virus.
Reason & solution: https://stackoverflow.com/a/52054580
Solution NOT working !

For meanings of <my_site_name>.spec, see ref cn (evernote bak)

some more tips: [full process in linux, csdn 2020]; [hidden imports, csdn 2020]

some more pitfalls (mentioning similar ones but different solutions):

Problem: MIME type not matched, so js not loaded/executed.
Reason: pyCharm send MIME correctly, but pyinstaller send .js files’ MIME type as plain/text together with nosniff, thus the browser treat js as plain text.
Solution: in each (or relevant) site folder’s settings.py:

import mimetypes
mimetypes.add_type("application/javascript", ".js", True)

nt solutions):

Problem: MIME type not matched, so js not loaded/executed.
Reason: pyCharm send MIME correctly, but pyinstaller send .js files’ MIME type as plain/text together with nosniff, thus the browser treat js as plain text.
Solution: in each (or relevant) site folder’s settings.py:

import mimetypes
mimetypes.add_type("application/javascript", ".js", True)

Problem: “Could not find a version that satisfies the requirement setuptools (from versions: none)”
If you want the build to “see” system installed packages, use pip install --no-build-isolation.