Monday, March 2

阴历新年( Lunar New Year) ,还是中国新年(Chinese New Year)?

上个星期一个越南裔同事请了一天假,在家庆祝中国新年(Chinese New Year)。其他同事还觉得奇怪:中国新年跟越南有什么关系?

亚洲汉字文化圈里,中国新年是头等大事,在一些国家里如越、新、马、韩等,这一天是法定假期,全国普天同庆;即使在这一天不是法定假期的国家,如日本、泰国等,民间的庆祝活动也很隆重。在这些国家里对这天的叫法固然不同,但是它们都按照中国的农历历法,在中国新年的同一天庆祝。

上星期美国国会议员孟昭文提出了为了尊重亚裔的传统习俗,在国会在这天放假的提议,深得众多在美国的亚裔民众的拥护。但是这个提议把这天叫做“阴历新年”(Lunar New Year),并明确说这一天是冬至之后第二个新月,这个说法不太正确。

中国农历历法并不是标准的阴历(月亮历)。每年(地球绕太阳一圈为一年)里,我们能看到的月亮圆缺约有12.4次,所以如果按照每12个阴历月为一年,第13个阴历月的第一天为新年的话,小数点后0.4的误差积累起来,几年之后你就会发现1月1日是在炎热的夏天!中国伟大的古代天文学家观察到这个误差,发现了“黄道”的规律,主要是地球自转夹角导致太阳直射点在北回归线和南回归线间转移的规律,将一年平均划24份,定下了24节气。比如说,北半球的中国大陆在太阳直射北回归线的那天,就是夏至,这天白昼最长;在太阳直射南回归线,对中国大陆偏射最厉害的那天,便是冬至。同样在西方,这两天也是很重要的日子,有研究认为在新石器时代,民众已经在这两天祭祀,祈福。总之,在太阳历的参照下,农历历法规定在每个冬至必须在11月(也称作“冬月”),每个月(月亮圆缺一次)大概有两个节气;如果哪一年(两个冬至之间)里,有13次月初,则肯定(起码)有一个月内只有一个节气,在把第一个这样的月定为闰月。这样造成的农历年每年长度不一样,比如,2013年有355天,2014年有384天(闰9月),2015年有354天,这三年平均下来有364天,就和太阳年的365.2天差不多了。遥想当年没有卫星技术,古代天文学家“不知所以然而知其然”,能算得这么精确,已经是很厉害!

所以说,中国农历是一个阴阳合历。把中国农历说成阴历"Lunar Calendar",把新年叫做阴历新年,这是孟议员的第一个错误。把这天叫做“冬至之后第二个新月”,这是第二个错误。2033年的冬至在12月21日,农历11月30日。第二天为只有一个节气的新月,即闰11月初一;2034年1月20日是冬至后的第二个新月,即12月初一;2034年2月19日,第三个新月,才是新年!

中国农历的计算,都是基于北京时间的,比如说今年2015年的冬至(太阳在北回归线的时候),是12月22日北京时间中午12点48分,这时候是加州的12月21日晚上8点48,东部时间晚上11点48。你能看见美国从夏威夷到纽约有5个时区,能造成很多的混乱,节气可能在不同一天,连每月的初一(新月的那天)都可以不一样,更别提距离地球另一边的北京了。今年2月份的新月就是在北京时间2月19日7点47分,纽约时间2月18日18点47分。全球都在2月19日庆祝中国新年,难不成美国国会要在18日(所谓的冬至之后第二个新月)庆祝?即使在亚洲的国家,只要与北京不在同一个时区,都可能碰到这样类似的问题,如果他们基于自己的时间计算农历的话。但是他们都根据历史渊源,按照中国“中原”的规矩,遵循中国所订的历法时间来度过这些传统节日。

所以,在美华人请不要再按照自己时区计算农历节日,中秋时就老老实实在北京时间满月的那天度过;新年时也按照北京时间新月的日子;农历新年还是叫做“中国新年”(Chinese New Year)最为妥当。

Wednesday, February 25

MySQL: Changing the size of the log file

I was told that the new MySQL (5.6+) is flexible about this thing, that is very good. I know people have been fighting with it for a long time. Since the Ubuntu 14.04 is still having MySQL 5.5 as default, I would like to write this down as a reference.

Situation: The default innodb_log file size is small, as 5M, if it is not specified in the /etc/mysql/my.cnf. As soon as you start mysql server, ib_logfile0 and ib_logfile1 are created in the /var/lib/mysql folder with 5M size. Then you can not simply change it.

With the default small log file size, you can run into
Error code 1206: The number of locks exceeds the lock table size.
when doing some complicate query, or loaddata big file.

In an ideal word (as in the new MySQL(5.6+)), you just need to add innodb_log_file_size=128M in the my.cnf file, restart the mysql server, the software would load the new configuration and resize the log file.

But in reality (as MySQL 5.5-), if you modify the my.cnf and restart, your mysql service can't never start, and the log tells you that setting is not consistent with the existing log file size. Sure, we know that and that is exactly what we are trying to do: resizing the log file. But what should we do now?

So you have to change back to innodb_log_file_size to 5M in order to proceed. Add
innodb_fast_shutdown = 0
in config file for purging any pending transaction into database. So the my.cnf has this 2 lines:
innodb_fast_shutdown = 0innodb_log_file_size   =  5M

At this time, you should able to start mysql service:
service mysql start
Good. Then we stop it properly:
service mysql stop
At this time, because all pending transactions are purged from the log file, we can safely remove the ib_logfile*:
mv /var/lib/mysql/ib_logfile* /tmp
Now you can edit the my.cnf file toset the innodb_log_file_size into the size you want, and remove (comment out) the shutdown parameter:
#innodb_fast_shutdown = 0innodb_log_file_size   =  128M
Then restart the service:
service mysql start

The ib_logfiles will be created with new size now.

Thursday, December 11

Amazon Fire TV Stick 记录

Jan 22, 2015 补记:看来这个Fire TV Stick不够力。在kodi/icefilms里跑一个视频停顿很厉害;我同时在电脑上以相同的 kodi/icefilms跑同一个视频,很流畅。
所以不建议购买。可能Fire TV会好一些?

再补记:上个月 app store里还没有,上星期找到了:Fire TV remote,完全可以替代原带的遥控器,还附有键盘。所以我装的bluetooth keyboard也退休了。



看到@Wang 的记录,忍不住于感恩节在Staples买了$25的Amazon Fire TV Stick,第二天又在Amazon下单买了个usb hub和bluetooth keyboard。两天后,Amazon已经送货来了,Staples直到前天才通知到货,整整10天啊。

电视机只有一个usb口,连接着一个室内天线。所以我要用这个usb hub才能同时插上室内天线和这个新的Fire TV Stick。

物理安装很简单,一个HDMI插口,一个usb接口,不可能搞错的。开机后它询问wifi密码,自动升级。在设置Parent Control时我设了个5数字的pin,后来发现这个Parent Control并不是我想象的“屏蔽任何PG视频”,而是“禁止任何视频、设置”,在随后的每个操作里都要输入pin,所以在输入20次之后我取消了Parent Control。

输入Amazon帐号时有一个bug。我逐个字母地输入我的 @fadshop.net 的帐号之后,它说email不合格。我输了三遍啊,三遍!后来在输入 @fadshop ,点击软键盘上的 .net ,它才认了。

用遥控器在软键盘上输入任何东西都是很痛苦的。Android手机可以安装一个 Fire TV app来代替遥控器,但是iPhone版本还没有发布。所以我前两天不是专门买了个bluetooth keyboard么?

因为我是Amazon Prime成员,登陆之后就出了所有能在电脑登陆Amazon Prime之后看的视频,速度都不错。

Sideloading:
按照 http://sideloadfiretv.com/sideload-apps-amazon-fire-tv-windows/ 安装了:
XBMC (kdoi - helix): 安装之后没有什么用处啊,电影、电视都是空的。难道要另外设置stream source?
Adobe Flash: working
Firefox: working
Android Terminal Emulator: 很温馨的窗口:
屏幕没有找齐,最左面的两个字母显示不出来。有空再调一下。

ES File Explorer: working,还没有看怎么连网络共享盘。
沙发管家:它是另一个桌面。从这个桌面可以再安装一些中文的媒体软件。我加上了全聚盒VST等。其中的视频有一些没有源,没法看。
Settings.adk:这个比较tricky。其它几个我都能找到官方站点来下载,这个我只能找到一个转手了几次的帖子,从Google Doc里下载,让我很不放心。安装执行后有很多settings可以设置,我找到Bluetooth的选项,错过了右上角的“Search Bluetooth”的按钮,颇费了点功夫,才找到这个按钮,把Bluetooth键盘顺利加上来了。
这是我专挑的有mouse touchpad的键盘,事实证明很好用,在用全聚合时选择视频很方便。
不方便的一点是“返回键”,要按Fn+1,不如遥控器上一个按钮就完成任务了。

然后回到Fire TV的Home屏幕,乱挑了一些youtube, tune-in, pbs-kids等app。所Sideload的那么多软件不能摆在这个Home屏幕上,而要选择Settings->Apps->Manage apps才能看到。

一个trick: 在全聚合里,可以设置“开机后执行”,这样每次开机,直接就看到中文视频全聚合了;要回到英文Fire TV 的Home的话,用遥控器的Home键马上到达。

xbmc是个播放器,需要装各种addon
 

Friday, May 9

One tip about logging.debug() in python

The idea of logging.debug( output string ) is that, if the logging system was set as higher level, the output string will be ignored, not being output to the logging system.

One problem, if the logging.debug() is calling some functions to format the output string, that functions ARE executed, no matter what the logging system is.

For example, I used this sentence to find out the size of the critical data object in the environment:
logging.debug("  size of dataprovider1 is: %s", sys.getsizeof(cPickle.dumps(dataprovider1)))
It works fine in debug environment; then when switching to production environment, even though the logging.level was set to INFO, this sentence was not being output to the log file, this function was still being executed, and an object of 90G size was created, and crashed the production system immediately.

Solution: use the isEnableFor() function of the logging.
if logging.root.isEnabledFor(logging.DEBUG):
    logging.debug("  size of dataprovider1 is: %s", sys.getsizeof(cPickle.dumps(dataprovider1)))

Monday, April 14

Add memory info to python logging: several options?

Our program is using a lot of memory. Too much of memory, maybe. So I want to write memory of current process in logging information.

Getting the memory size of current process is the easy part (in Linux):
resource.getrusage(resource.RUSAGE_SELF).ru_maxrss
I can think of several ways to add this into logging system: Adding a handler (derive the handler from either StreamHandler or FileHandler), or change the Formatter. Finally I decided to add a filter:
class MemFilter(logging.Filter):
    def filter(self, record):
        record.mem = resource.getrusage(resource.RUSAGE_SELF).ru_maxrss
        return True
logging.basicConfig(format='[%(levelname)s][MEM:%(mem)s] [%(asctime)s]:%(message)s')
logging.root.addFilter(MemFilter())

Case closed.

Wednesday, April 2

Display current time for each command, in Python Interactive Interface

I like making the prompt sign (ps) into showing current time, so that I know exactly what time I executed a command, and what time it is completed. I made that for DOS command, Linux terminal, and MySQL terminal. Here I will show you how to do that in Python Interactive Interface (Python Interpreter).

Here is what it is after setting:



The first thing, setup a $PYTHONSTARTUP environment variable to point to a python file location. Exactly how to do that depends on your operating system. Now matter how it is done, the file it points to will be executed when "python" (Python Interactive Interface) is called.
In my Ubuntu, execute
export PYTHONSTARTUP=~/.pythonstartup.py
Next step, in my home folder "~", I created the file .pythonstartup.py as:
# filename: .pythonstartup.py
# after setting $PYTHONSTARTUP=~/.pythonstartup.py, this file will be executed before python interactive interface is open
import sys, time
sys.ps1 = time.strftime("%A %H:%M:%S") + ">>>"
Pretty simple, then the prompt sign (I guess that is the full name of "ps) will be current time, as "Wednesday 16:49:51" showing in the above screenshot.

PS: the way for showing the same information in linux command line (Ubuntu) is to modify the PS1:
PS1='${debian_chroot:+($debian_chroot)}\[\033[01;32m\]\u@\h\[\033[00m\]:\[\033[01;34m\]\w\[\033[00m\] [\D{%a} \t]\$ '
(add this in .bashrc)

For mysql client tool, add this line under [mysql] of the ~\.my.cnf:
prompt=\\u@\\d [\\w \\R:\\m:\\s]>



Friday, March 28

海角七号:看见月亮渐渐光

海角七号里,老头用威逼利诱,走关系进了乐队,好快活地在这个摇滚舞台弹着古琴唱:
看见月色渐渐光
有话对你讲
妹妹你啊想一想
...
根据百度,这首歌是《南都之夜》
《南都之夜》(曲:许石 词:郑志峰)
我爱我的妹妹啊 害我空悲哀
彼当时在公园内 按怎你甘知
看见月亮渐渐光 有话想要问
请妹妹你想看觅 甘苦你甘知
我爱我的妹妹啊 相招来[辶日]迌
黄昏时在爱河边 想起彼当时
双人对天有立誓 我无娶你无嫁
亲像风雨浇好花 何时再相会
双人对天有立誓 我无娶你无嫁
亲像风雨浇好花 何时再相会
这首《南都之夜》创作于1945年,而就是在1945年发生了一件台湾历史上很重要的一件事,就是日本战败,台湾光复,台湾从此结束了50年的日据时期,回归祖国,而这首歌正是创作在台湾光复后不久。
关于这首歌呢,其实还有一段历史,1945年国民党接收台湾时﹐全面禁止日本歌﹐但是由于日本统治时期限制台湾歌(包括其他台湾的文化)﹐故写歌在当时是无法维生的﹐所以很多艺人都纷纷改行或是转唱日本歌曲,以至于禁止日本歌后台湾同胞每日听来听去的都是些军歌或进行曲﹐后来忽然有首歌曲就在大街小巷流行起来﹐这首歌呢就是《南都之夜》。
在1959年的电影《空中小姐》里,同样的曲子,被唱成了《台湾小调》:



当然,现在更让这首曲子广为流传的是李克勤唱的《旧欢如梦》:

当年相恋意中人,大家性情近;早种爱根极亲密,心心相印互信任。
月底花间相偎依,共喜有缘份;恩爱百般愿比翼,痴心一缕共订盟。
喜逢知己倍精神,内心快乐无憾;朝晚眷恋共欢聚,天天相见互慰问。
立心栽花花不香,仲反惹仇恨;只怨爱海起风波,一朝生变断爱盟。
恩情于今化烟云,未许再续情份;空有爱丝万千丈,可惜都已尽化恨。
枉抛相思枉痴恋,恨卿心太忍;只有叹息旧欢似梦,早经消散莫再寻。
早经消散莫再寻……


喇叭和上海舞步,都是我的最爱。

Saturday, March 22

并非敌人Not the enemy

译者前言:有感于国内正在收紧网络言论的控制,罔顾网民(人民)的隐私/权利,我把2600去年夏天的编辑弁言Not the enemy的一段翻译一下。黑客只是好奇于手册中没有记载的漏洞,热爱追求自由的一群人。他们不是敌人。

黑客们发现问题,告诉大众。他们是奥巴马所宣誓要支持的开放环境的最典范例子。他们不是从公司阴谋中取利的坏人,也没有满世界送垃圾邮件,或用病毒和蠕虫制造混乱。许多年来,媒体造成这样的形象:任何跟网络或计算机有关的坏事,都是黑客做的。讽刺意味的是,这使得那些做了那些坏事的人很骄傲地称自己为黑客。其实他们并不是。只要瞥一下本刊物(2600)上这些持续的对话,任何一个外行都能看出真正的黑客对于做坏事是很严肃的。如果给每个用键盘来做坏事的人这个标签,


-- 2014/03/22 注:这是2010年4月敲下来的,不知为何却没有发出去。

Labels: , ,

Friday, March 21

A bug, definitely a bug, in MySQL

First of all, I have a very complicate query to insert from select group by on duplicate update. So I need to add "select * from () " from the group by subquery, then this bug is shown: The subquery result is different from "select * from (subquery)"!

The subquery is also very complicate, but I can simplify it as following:
create table test(
firstdate datetime default null,
lastdate datetime default null
);
I need to find the last date of this table, but either of the 2 fields can be null. So the subquery I came up with is:
select GREATEST( IFNULL(MAX(firstdate), 0), IFNULL(MAX(lastdate), 0)) from test;
That works perfectly. You can insert some dates for testing:
insert into test(firstdate) values('2014-03-08');
insert into test(firstdate, lastdate) values(' 2012-02-01', '2013-03-08');
Then the query will have expected result:
'2014-03-08 00:00:00'
Here is the wired part: Adding select * from () will have different result!
select * from (
        select GREATEST( IFNULL(MAX(firstdate), 0), IFNULL(MAX(lastdate), 0)) from test) t
'2014-0'
Work-around:
Change the "0" in subquery to a datetime string such as '0000-00-00 00:00:00':
select GREATEST( IFNULL(MAX(firstdate), '0000-00-00 00:00:00'), IFNULL(MAX(lastdate), '0000-00-00 00:00:00')) from test;
Then select * from (subquery) will have the expected result.

Analysis:
Looks like that because of the "0" value, the IFNULL changed the result column somehow into a string? Even if all the firstdate and lastdate are filled (there is no null value in the table), the result is still the same.
But if the result column property is modified, why the subquery itself works fine?

MySQL 5.5.

Reported as  Bug #72098

Friday, March 14

Using python to execute a .sql file for mysql

The task is to execute one import.sql file to import several data files into mysql. The trick is that it is known some files will be missing, so the execution of the import.sql will have error, but we want to ignore the error, import whatever exists in the harddrive.

There are two ways to do that:

1, Execute mysql command line, feed in the file as pipe input:
    f = open("import.sql")
    process = Popen(['mysql', '--force'], stdout=PIPE, stdin=f, stderr=PIPE)
    out, err = process.communicate()
The trick is to use '--force' argument of mysql to ignore the error. This is equivalent to:
mysql --force < import.sql
The err will have the error output for missing data file, then you can write that into log file.
2, Execute mysql  command line client, then execute "source import.sql":
    from subprocess import Popen, PIPE
    process = Popen('mysql', stdout=PIPE, stdin=PIPE, stderr=PIPE)
    out, err = process.communicate(input="source import.sql")
Last time I checked, this way was not working very well. It always failed at the missing data file step.
Reference: http://bugs.mysql.com/bug.php?id=533 : "--force is intended to be used in Batch Mode only."

Labels: