Skip to main content

Posts about python

RSS feed

devpi-server

devpi-server is a server for private package indexes and PyPI caching.

build a docker container

make a dockerfile

FROM debian:buster-slim

ENV DEBIAN_FRONTEND noninteractive
RUN apt update && \
    apt install --no-install-recommends -y \
    python3-pip python3-pip python3-setuptools python3-dev build-essential libffi-dev && \
    pip3 install devpi-server

EXPOSE 3141
VOLUME ["/var/cache/devpi"]

CMD chmod 777 /var/cache/devpi && \
    devpi-server \
        --serverdir /var/cache/devpi \
        --host 0.0.0.0 --port 3141

build an image

$ sudo docker build --build-arg http_proxy=http://192.168.xxx.xxx:3142/ -t devpi-server:buster-slim .
$ sudo docker tag devpi-server:buster-slim devpi-server:latest

run a container

$ sudo docker run --rm -d -p 3141:3141 -v /mnt/devpi:/var/cache/devpi devpi-server:latest

test the address and port

$ curl http://192.168.xxx.xxx:3141/

how to use devpi-server

specify it in a command line

$ pip3 install --trusted-host 192.168.xxx.xxx --index-url http://192.168.xxx.xxx:3141/root/pypi matplotlib

or 

$ PIP_TRUSTED_HOST=192.168.xxx.xxx PIP_INDEX_URL=http://192.168.xxx.xxx:3141/root/pypi pip3 install matplotlib

for docker build, write below in a Dockerfile

ARG PIP_TRUSTED_HOST
ARG PIP_INDEX_URL

then specify devpi-server at build-arg parameter in docker run command line.

$ docker build --build-arg PIP_TRUSTED_HOST=192.168.xxx.xxx --build-arg PIP_INDEX_URL=http://192.168.xxx.xxx:3141/root/pypi -t imagename .

json in python

dictionary

add key:value pair to a dictionary

>>> newdict = {}
>>> newdict['foo'] = "bar"
>>> newdict
{'foo': 'bar'}

get all keys or values of a dictionary

>>> newdict.keys()
['foo']
>>> newdict.values()
['bar']

get a value for a key from a dictionary

>>> newdict['foo']
'bar'
>>> newdict.get("foo")
'bar'

add a dictionary as a value

>>> newdict0 = {}
>>> newdict0["mykey"] = newdict
>>> newdict0
{'mykey': {'foo': 'bar'}}

get a value

>>> newdict0['mykey']['foo']
'bar'

add a list as a value

>>> newdict0["mykey"] = [newdict, {'hoge': 'piyo'}]
>>> newdict0
{'mykey': [{'foo': 'bar'}, {'hoge': 'piyo'}]}

get a list

>>> for dict in newdict0.get('mykey'):
...     print dict
...
{'foo': 'bar'}
{'hoge': 'piyo'}

json

dump dictionary as json format string

>>> import json
>>> mystr = json.dumps(newdict)
>>> mystr
'{"foo": "bar"}'

load json format string to dictionary

>>> mydict = json.loads(mystr)
>>> mydict
{u'foo': u'bar'}
>>> mydict.get("foo")
u'bar'

To show non-ASCII characters, we can use ensure_ascii=False option, otherwise the output are escaped with \uXXXX sequences

$ python26
>>> import json
>>> mystr = '{"foo":"ばぁ"}'
>>> mydict = json.loads(mystr)
>>>
>>> print(json.dumps(mydict))
{"foo": "\u3070\u3041"}
>>>
>>> print(json.dumps(mydict, ensure_ascii=False))
{"foo": "ばぁ"}
>>>

If locale is not UTF-8, we will met UnicodeEncodeError when it prints UTF-8 strings to stdout. In this case, we can use encode() to avoid the error.

$ LANG=C python26
>>> import json
>>> mystr = '{"foo":"ばぁ"}'
>>> mydict = json.loads(mystr)
>>>
>>> print(json.dumps(mydict))
{"foo": "\u3070\u3041"}
>>>
>>> print(json.dumps(mydict, ensure_ascii=False))
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
UnicodeEncodeError: 'ascii' codec can't encode characters in position 14-15: ordinal not in range(128)
>>>
>>> print(json.dumps(mydict, ensure_ascii=False).encode('utf-8'))
{"foo": "ばぁ"}
>>>

write json to a file

mystr = '{"foo":"ばぁ"}
mydict = json.loads(mystr)

with open('new.json', 'w') as f:
    json.dump(mydict, f)

read json from a file

with open('new.json') as f:
    myjson = json.load(f)
    print(json.dumps(myjson, ensure_ascii=False).encode('utf-8'))

sample script

using urllib2 library

similar as above but using reauests library