Monday, November 2

VPN使用问题



不断有“私人使用VPN被公安问询处罚”的消息。例子:


我们就拿这个《计算机信息网络国际联网管理暂行规定》来说说。


中国的政府网站很乱。最简单的例子,首先就是你很难找到发布这个规定的部门“国务院信息化工作领导小组”的主页。即使曲折找到了这个部门现在的名字“中央网络安全和信息化委员会”,在它的主页 http://www.cac.gov.cn/ 上搜索这个规定,也找不到它。网络搜索这个规定首先拿到的是百度百科里的词条,然后才在一些地方政府部门看到这个条文。政府条文规定由本部门发布在自己主页上,本来是很简单的事情,不知道为什么中国就这么难做到。前几天美国国安新出的 H1B的政策,就很容易找到它网站里公布的整整100页的全文: https://www.dhs.gov/sites/default/files/publications/20_1028_uscis_h-1b-registration-selection-by-wage-levels-nprm-508.pdf



且让我们相信这些地方政府部门的网站是可信的,访问河北通信局网站 https://hbca.miit.gov.cn/html/detail/201807/94f192dee566b018e0acf31e1f99a2d9.html 能够看到:


第六条计算机信息网络直接进行国际联网,必须使用邮电部国家公用电信网提供的国际出入口信道。

任何单位和个人不得自行建立或者使用其他信道进行国际联网。


第十四条违反本规定第六条、第八条和第十条的规定的,由公安机关责令停止联网,给予警告,可以并处15000元以下的罚款;有违法所得的,没收违法所得。



现在的处罚规定说“擅自建立非法信道进行国际联网”,就是违反了第六条中的“必须使用国家公用电信网提供的国际出入口信道”。这是错误的。


同一个河北通信局的页面下面,还有同一个部门同时发布的这个规定的《实施方法》,其中的第三条写得很清楚:

第三条本办法下列用语的含义是:

(三)国际出入口信道,是指国际联网所使用的物理信道。



现在,让我画图解释一下国际联网、信道,和VPN的运作。


首先,画个中美网络拓扑图:



中国人要访问外国(美国)网站时,使用国家公用电信网提供的信道,能够访问到美国:


根据“相关规定”,这个信道依法屏蔽了一部分外国网站。在图中,红色的点是可以访问的,黑色的不能访问。这就是所谓的“墙”,正式名字是中国的“网络防火长城”。网上的分析已经很多,而中文维基中对它的总结很精到,请让我拷贝/摘录在这里:


网络防火长城是对中华人民共和国政府对中国大陆境内的互联网所创建的审查系统(包括相关行政审查系统)的统称。

世界其他一些国家也存在网络审查,不过其审查对象、规模、执行主体等均与中国的审查机制有着相当大的不同(参见:互联网审查),例如仅止于金融洗钱、国际诈骗等犯罪行为,或者仅审查儿童色情相关。而防火长城的作用是监控所有经过国际网关的通讯,对认为不符合中共官方要求的传输内容,进行干扰、阻断、屏蔽。

北京当局一直没有正式对外承认防火长城的存在,只是宣称“依法对网络进行管理”。


至于哪些海外网站被墙所屏蔽,哪些可以开放,政府没有具体地说法,只是泛泛地说不符合中国法律要求的就屏蔽。刚才提到的中文维基页在被屏蔽之列,不少国外大学的官方网站页被屏蔽(what???) 。屏蔽列表是不公开存在的,你只能在国内尝试访问,成功了就是成功了,失败了就是失败了,没有查询、申诉的渠道。



回到我们要讨论的事情上:上面所说的是通过“国家公用电信网提供的信道”访问国际网络。那么怎样才是“使用其他信道进行国际联网”(第六条中禁止的行为)呢?


在深圳罗湖,高楼耸立。有些高楼顶上有锅状天线,对着香港。是的,这些天线有些是合法批准的,但也有一些是私下成立,或者占用合法批准的频道,私下与香港联网:



这是我能想到的第一个“使用其他通道进行国际联网”的方式,另一个方式是使用亿龙。马斯克的星联,只要在他主页上登记注册,在后院放一个直径半米的天线锅,就可以联网。不过后面这个也是中国明确禁止的:1990年广电部,公安部,国家安全部一起发布的《卫星地面接收设施接收外国卫星传送电视节目管理办法》禁止了民间使用这种天线锅。其实要较真的话,用这个锅来接受星联网络并非传送电视节目,但是又落到了这个《计算机信息网络国际联网管理暂行规定》的管理范围内。


这两个方法就是《规定》的《实施办法》里第三条所说的:

国际出入口信道,是指国际联网所使用的物理信道。


一般宅男在电脑前,实在没有建造、使用这样的通道进行国际联网的机会。那么大家常说的“翻墙”是怎么做的呢?

当用户如图(2)般访问了美国的红色站点(防火墙所允许的站点)之后,使用它为跳板,委托它去拿黑色站点(被防火墙屏蔽的站点)的内容,送回来给用户:


这就绕过了防火墙的屏蔽,访问到先前无法访问的黑色站点。早期的代理服务器、现在的VPN,Shadowsocks, 都是使用“可访问服务器”为跳板达到翻墙的效果。


讨论到这里,总结一下:

1,“翻墙”所使用的是合法的国家公用电信网提供的信道,达到可允许访问的红色站点;

2,个人访问国外(可允许)站点之后,跟那个站点如何协调、如何交易,是个人与站点的事务,国家权力不应该介入;

3,如果个人使用网络访问、传播违法内容(黄色、暴力、反动等内容),由另外的法律监管;

4,如果访问屏蔽站点(黑色站点)是非法的,国家必须公布这些站点,让人有法可依。事实是国外大多数的著名媒体网站都被屏蔽,不少国外大学的网站页被屏蔽,没有了这些网站,正常的网络生活都很受挫折。



结论:使用《计算机信息网络国际联网管理暂行规定》第六条“任何单位和个人不得自行建立或者使用其他信道进行国际联网。” 对VPN的使用进行处罚是一个错误的方式,因为VPN所使用的是合法的信道。


Sunday, November 1

金木水火土

 晚饭前,我在白板上写下“金 木 水 火 土” 五个字,准备在女儿过来后,介绍中国的这五大元素,然后中国先民是如何发现天上五颗星星的痕迹跟其他恒星不一样,从而用这五个元素给这些星星起名字。


女儿过来了,看了一眼,我让她读一下这几个字。她都会,很轻松读下来了,然后嘟囔了一句:“为什么第一个字‘金’比其他几个都复杂很多?”


电光石火间,我有了很完美的答案:

“因为其他四个都是人类很早就发现了,相应的文字也发明了给它们;人类后来才发现了金属,所以这个‘金’字就比其他字都复杂了。”

Labels: , ,

Wednesday, September 16

Setting GoDaddy hosted site to use Python3/Flask

I planned to actually use my hosted site of fadshop.net in GoDaddy, so last time I explored using C# in Windows hosted site (Using Visual Studio to build a website, and publish it in the GoDaddy hosting site). 

Yes, it was working, but my patience in Windows is thin. Now I switched the site back to Linux.

Again, the software in the GoDaddy hosted server is outdated. Last time, in the Microsoft hosted site, the server only has ASP.NET, doesn't have ASP.NET Core which was published since 2016. Now, the python version is 2.6.6, which was released in 2010. 

So the first thing I did was to find a way to run Python 3 for my website that I planned to build.

After reading dozens of pages about this topic, I found the method was to compile and install python3 under your own folder as non-root user. At first I thought that was a hack, until I found a blog page of GoDaddy:  How to install and configure Python on a hosted server , using the same method, then I was sure that is the right/correct/plausible way of getting things done.

My main reference pages are:
https://towardsdatascience.com/installing-python-3-and-flask-on-godaddy-1635fe6f24bc
https://www.godaddy.com/garage/how-to-install-and-configure-python-on-a-hosted-server/
https://www.reddit.com/r/linuxquestions/comments/c5wxh0/help_with_error_on_install_of_python37_from/

All these pages ( and other Googled pages) gave similar solutions but all of them failed during my practice. So I have to consult dozens of other pages (including C compiler parameter setting information)  and add my knowledge into this practice to make it work. Hopefully this is also helpful for you.


First of all, this is a clean GoDaddy hosted "Economy Linux Hosting with cPanel". I think it is CentOS 6 or 7. You can either turn on "SSH access" in your hosting server page, or get to the Terminal in the cPanel.

I have been using Linux for a long time, and usually I am the root/admin of the server, and the software can be easily installed using yum or apt-get. Now with a non-root account, things are very different. The idea is to install all software/library into one folder under this non-root account. In here I am using  "~/.python/" folder,  and it is also /home/zwsjyo79b9/.python, $HOME/.python in the following scripts:

Step 1, set path, library path environment variables:
 
export PATH=$PATH:~/.python/bin
export LIBRARY_PATH=$LIBRARY_PATH:~/.python/lib:~/.python/lib64
export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:~/.python/lib:~/.python/lib64
export C_INCLUDE_PATH=$C_INCLUDE_PATH:~/.python/include
export CPLUS_INCLUDE_PATH=$CPLUS_INCLUDE_PATH:~/.python/include
export PKG_CONFIG_PATH=~/.python/lib/pkgconfig:$PKG_CONFIG_PATH
export LD_RUN_PATH=~/.python/lib:~/.python/lib64
You can put this into your ~/.bash_profile for future use. The lib64 in this page is my invention and the key difference between this post and other people's pages. Some of the software created library files in this folder. It might be a new thing, so the other pages didn't mention it, but without it, the python compilation will fail miserably.


Step 2, before compiling python3, we need to have 3 things ready:
1. openssl package. Without it, the pip of the new python3 will not work. You will get this message when running pip:
pip is configured with locations that require TLS/SSL, however the ssl module in Python is not available.
A python without pip is useless. I blame python for letting itself compiled at all without openssl. It wasted me a lot of time on that.
2, libffi. Without it, the python package will failed with this message when installing some package like jasonpickle or some other normal operation:
ImportError: No module named '_ctypes'
Again, I blame python for letting itself compiled without this package.
3, sqlite. I will use sqlite in my python program, and I can't just do "pip install pysqlite" at that time. Although this GoDaddy server already has sqlite running, it is required to compile and install it (with libraries) for the compilation of python3 to include sqlite module.

So now I create one folder ~/tobeinstall/ , and google for these 3 packages (openssl, libffi, sqlite) and downloaded their source code packages. At the time of this post, they are retrieved by:
wget https://www.openssl.org/source/openssl-1.1.1g.tar.gz
wget https://github.com/libffi/libffi/releases/download/v3.3/libffi-3.3.tar.gz
wget https://www.sqlite.org/2020/sqlite-autoconf-3330000.tar.gz
then "tar xzvf xxxx" to unzip them into corresponding folders.
Execute these commands to compile and install them:


cd openssl
CFLAGS=-I/home/zwsjyo79b9/.python/include LDFLAGS=-L/home/zwsjyo79b9/.python/lib ./config --prefix=$HOME/.python
make
make install

cd ../libffi
CFLAGS=-I/home/zwsjyo79b9/.python/include LDFLAGS=-L/home/zwsjyo79b9/.python/lib ./configure --prefix=$HOME/.python
make
make install


cd ../sqlite3
CFLAGS=-I/home/zwsjyo79b9/.python/include LDFLAGS=-L/home/zwsjyo79b9/.python/lib ./configure --prefix=$HOME/.python
make
make install

The idea is to use --prefix=$HOME/.python to specify the installation folder. 
Note: The libffi will install some library files in the ~/.python/lib64 folder. This is not mentioned in other discussions about compiling python3.


Step 3, with all these packages installed, we can start getting python source code:
wget https://www.python.org/ftp/python/3.8.5/Python-3.8.5.tgz
and untar it. Before compilation, we need to modify one file in the unzipped folder:
vi Modules/Setup


Uncomment the lines about SSL, and give the correct path to SSL

Now with this specific command, you can start building:
CFLAGS=-I/home/zwsjyo79b9/.python/include LDFLAGS="-L/home/zwsjyo79b9/.python/lib -L/home/zwsjyo79b9sc/.python/lib64" ./configure --prefix=$HOME/.python --enable-loadable-sqlite-extensions  --with-openssl=/home/zwsjyo79b9/.python 

The configuration process will take several minutes to complete. Make sure you see:

checking for openssl/ssl.h in /home/zwsjyo79b9/.python... yes
checking whether compiling and linking against OpenSSL works... yes
checking for X509_VERIFY_PARAM_set1_host in libssl... yes
That is the proof that the openssl is correctly installed, and that is the guarantee that pip will perform correctly.

Now it is time to compile it:
make
Check the output like this:


Make sure "sqlite" and "ctypes" are not in the "The necessary bits to build these optional modules were not found" group.

Now you can do "make install" to complete the installation. With the "export PATH=$PATH:~/.python/bin"  you put in the beginning of this tutorial, you should be able to execute "python3" now.


Now we get to the part to actually use it for your application. First of all, my suggestion is to use a subdomain to play with this, keep to main site as default. To do that, you need to create a subdomain in your cPanel. For example, when I create a "deepparser" subdomain, the system will automatically set the default file path as "/home/zwsjyo79b9/public_html/deepparser.fadshop.net".  The rest of this tutorial are mostly copied from the second part of  Installing Python 3 and Flask on GoDaddy with some modifications:

public_html/deepparser.fadshop.net/
├── .htaccess
└── cgi-bin/
├── app.cgi
├── app.py
├── templates/
│ └── home.html
└── venv/



Create a cgi-bin folder in this subdomain, then execute:
pip3 install virtualenv
virtualenv -p /home/zwsjyo79b9/.python/bin/python3 venv    # make sure the created virtual env is using the lately installed python3 environment
source venv/bin/activate
pip install Flask                    # if you have other package to install, now it is the good time
deactivate

Now create the cgi-bin/app.cgi file with this content:

#!/home/zwsjyo79b9/.python/bin/python3
import os
import sys
sys.path.insert(0,
                '/home/zwsjyo79b9/public_html/deepparser.fadshop.net/cgi-bin/venv/lib/python3.8/site-packages')
from wsgiref.handlers import CGIHandler
from app import app
class ProxyFix(object):
    def __init__(self, app):
        self.app = app
    
    def __call__(self, environ, start_response):
        environ['SERVER_NAME'] = ""
        environ['SERVER_PORT'] = "80"
        environ['REQUEST_METHOD'] = "GET"
        environ['SCRIPT_NAME'] = ""
        environ['QUERY_STRING'] = ""
        environ['SERVER_PROTOCOL'] = "HTTP/1.1"
        return self.app(environ, start_response)
if __name__ == '__main__':
    app.wsgi_app = ProxyFix(app.wsgi_app)
    CGIHandler().run(app)
app.wsgi_app = ProxyFix(app.wsgi_app)
CGIHandler.run(app)
#Note: The origin page has error in the 5th line about sys.path.insert(). Information in this page is correct.
And create a simple Flask application in cgi-bin/app.py:
from flask import Flask, render_templateapp = Flask(__name__)@app.route('/')
def home():
return render_template('home.html')

Now create a cgi-bin/templates/home.html  and type anything inside. We are just trying to prove the content in this file will be served when visitor visit the subdomain "http://deepparser.fadshop.net/"

Now comes to the last file, .htaccess in the subdomain folder. For example, in my server, this file is /home/zwsjyo79b9/public_html/deepparser.fadshop.net/.htaccess
The content:

Options +ExecCGI
AddHandler cgi-script .py
RewriteEngine On
RewriteCond %{REQUEST_FILENAME} !-f
RewriteRule ^(.*)$ /home/USERNAME/public_html/python/cgi-bin/app.cgi/$1 [L]
Now we have all the files, but one more thing to do: Set all the files as 755, executable, but not writable for other users. 
chmod 755 *
This is required for the system to work.

If you have to understand the logic, at first the .htaccess in the main folder of the subdomain has a ReWriteRule to redirect all request (link) into the cgi-bin/app.cgi , then the app.cgi call the newly installed python3 with the library of the virtual environment (cgi-bin/venv/lib/python3.8/site-packages) to handle the request, and pass the request to app.py, which is a standard Flask application.

Now the server (this subdomain) is ready. Fire up your browser to visit it:
 














Wednesday, September 9

从世界末日到烟花扬州

 今天天象很奇特,中午时分,外面惨黄一片,很像以前暴雨之前飞沙走石的颜色,其实一点风都没有。听说是山火的烟雾升上高空,把太阳都遮住了。总之,一副世界末日的惨象。


说起“世界末日”,我就跟老婆吹嘘:“你知道‘世界末日’怎么翻译?”

“Then end of the world?”

“不是,在西方文化里,应该翻译成 Judgement Day.”

“为什么?”

圣经经常说起,有一天,神会回到我们人间,到时谁该上天堂,谁该下地狱,就会被一一裁决。圣经里还用了整整一本书《启示录》(Book of Revelation)来描写这一天的各种异象,各种灾殃。

这一天是这个世界的最后一天,然后所有人就转换场地了。

施瓦辛格的《终结者》2 就是以“Judgement Day”为名字,说在原本历史上将要出现的核弹毁灭,而它与主角将要改变这段历史,改变人类可怜的命运。


总之,需要对西方文化有足够的了解,才能理解从“世界末日”到“Judgement Day”的翻译。与之可喻美的一个翻译是将“烟花三月下扬州”的三月翻译成April。 啊?词典上,三月不应该是March吗?这时候你就需要理解古代农历的三月是现代的春光明媚的四月份,而不是春寒料峭的三月。必须翻译成April,才能让人享受明媚春光、花团锦秀的扬州时光。


说完文化和翻译,循例放一首应景的《直到世界末日》,歌手齐秦:


这首歌的原版是 "A Spaceman Came Travelling"

Tuesday, September 8

学西班牙语

 这段时间来,我一直用 duolingo.com 自学西班牙语。


这天和妻子出去跑步,在街头经过两个路人,他们正在聊天,用的正是西班牙语!我很激动,跟妻子说:

“我学了这么久的西班牙语,终于有了用场!我听出来了,刚才他们聊天用的是西班牙语!”


妻子也替我高兴:“很高兴你的学习派上了用场!他们说了什么?”

我按耐不住自己的激动,说:“他们说的是西班牙语!”


妻子进一步问:“你听见了什么?”

我很耐心地又说一遍:“我听出来了,他们说的是西班牙语!”


Monday, August 31

关于回车符、换行符

 

打字机(这个组里用过原始机械打字机的人不多)上有两个机械装置,一个是回到句首,另一个是换行,真正使用时这两个机械装置是联合在一起的,成为回到句首并换行,就是 ENTER 键的作用。这就是我们所说的"回车".

 

回到句首在英文里是 Carriage Return, 就是让那个附带着纸的转盘回到原始位置,打字口对着行首

换行在英文里是"Line Feed",让纸往上走1行,打字口就对着下一行。有时也可以翻译成进纸

所以有时你能看到 CRLF,就是指这两个操作。



在电脑发明之后,国际标准设计了 ASCII表,比如 65来表示A66表示B;其中还用13来表示换行 Line Feed, 10来表示回到句首 Carriage Return。在Word, textpad上,只要发现文件里这字节是1013在一起,就从下一行开头显示下一个字节。

这样的不能直接显示,但是有意义的字符我们叫做控制字符

Unix/Linux/Mac里,设计人员偷懒,因为这两个字节总是连在一起的,那么就用其中一个就足以表示换行(并回到句首)了,所以他们只用 ASCII 10 Carriage Return 来做这件事情。Linux是爽了,但是跟Windows交换文件时就经常出现混乱,所以还特地写了两个程序来替换回车符,一个叫做 dos2unix,另一个叫做 unix2dos (很没有创新感)。



总说“ASCII 10 Carriage Return ” “ASCII 13 Line Feed ” 很麻烦,写在程序上也乱,所以我们给了这两个字符别称,第一个叫做 '\n', 第二个叫做 '\r' 。在C语言里,单引号表征字符,比如 char c='\n';  大家一看就知道这是一个ascii 10 的控制字符,在显示时应该走到下一行;而如果用 string s="\n"; 这就是两个字符,分别是 ascii=92 '\' ascii=110'n' python语言里没有单引号和双引号的区别,所以它规定:如果 \ 后面带着有意义的字母,比如 \r, \n, \t, \f ,它就认为这是控制字符,就把 "\" 和后面的字母 (这两个字符)记录成一个控制字符的ascii码。 如果你要拒绝这个转义,在代码里你要写两个斜杠。例子:





>>> print("a\nb")

a

b

>>> print("a\\nb")

a\nb



能看到 "a\nb" 和 "axb"一样都是3个字节。

希望从这个例子里你能看出:我们必须分清楚,当我们说 "\n"的时候,是表示一个字节的控制字符?还是表示能够看得见得两个字符!