分类代码人生下的文章

安装Selenum的Chromedriver

作者: Jimmygao
时间: 2014-12-21
分类: 代码人生
评论

首先我们需要下载Chromedriver：

http://chromedriver.storage.googleapis.com/index.html

截图如下：

Chromedriver

下载最新版本（目前2.13）

将下载下来的文件移至/usr/bin目录下

完成！

参考链接： http://damien.co/resources/how-to-install-chromedriver-mac-os-x-selenium-python-7406

git fetch和pull的区别

作者: Jimmygao
时间: 2014-12-11
分类: 代码人生
评论

转自 http://blog.csdn.net/hudashi/article/details/7664457

Git中从远程的分支获取最新的版本到本地有这样2个命令：

1.git fetch：相当于是从远程获取最新版本到本地，不会自动merge

git fetch origin master
git log -p master..origin/master
git merge origin/master

以上命令的含义：

首先从远程的origin的master主分支下载最新的版本到origin/master分支上然后比较本地的master分支和origin/master分支的差别最后进行合并

上述过程其实可以用以下更清晰的方式来进行：

git fetch origin master : tmp
git diff tmp 
git merge tmp

从远程获取最新的版本到本地的test分支上之后再进行比较合并

2.git pull：相当于是从远程获取最新版本并merge到本地

git pull origin master

述命令其实相当于git fetch 和 git merge 在实际使用中，git fetch更安全一些因为在merge前，我们可以查看更新情况，然后再决定是否合并

linux无界面(headless)使用selenium抓取数据

作者: Jimmygao
时间: 2014-12-09
分类: 代码人生
6 条评论

问题

老高最近遇到一个需求，linux\centos下，使用selenium技术抓取数据。本来很简单的问题，但是由于内存限制，安装X window不现实，所以一个BT的想法诞生了，是否可以在centos命令行界面运行一个虚拟的桌面，然后使用selenium控制Firefox浏览器完成一些操作，Firefox运行在虚拟的桌面中，一切操作都在命令行中完成。

Google之，发现了Xvfb，他可以新建一个虚拟的X窗口，再配合python的pyvirtualdisplay，简直就是神器！

安装

centos下：

# 安装Xvfb和pyvirtualdisplay
yum install xorg-x11-server-Xvfb
pip install pyvirtualdisplay

安装firefox和selenium

yum install firefox
pip install selenium

代码

from pyvirtualdisplay import Display
from selenium import webdriver

display = Display(visible=0, size=(800, 600))
display.start()

browser = webdriver.Firefox()
browser.get('http://www.google.com')
print browser.title
browser.quit()

display.stop()

参考网站：

http://selenium-python.readthedocs.org/en/latest/getting-started.html http://nullege.com/codes/search/selenium.webdriver.Remote.find_elements_by_class_name http://www.opsview.com/forum/opsview-core/how-do-i/how-do-i-install-selenium-centos-server https://gist.github.com/textarcana/5855427 http://scraping.pro/use-headless-firefox-scraping-linux/ http://serverfault.com/questions/363827/how-can-i-run-firefox-on-centos-with-no-display https://realpython.com/blog/python/headless-selenium-testing-with-python-and-phantomjs/ https://pypi.python.org/pypi/selenium

http://selenium.googlecode.com/git/docs/api/py/selenium/selenium.selenium.html#module-selenium.selenium

http://www.ibm.com/developerworks/cn/opensource/os-php-designptrns/ http://www.cnblogs.com/fnng/p/3230768.html http://www.cnblogs.com/fnng/p/3157639.html http://www.cnblogs.com/fnng/p/3157639.html

mysql开发常用SQL

作者: Jimmygao
时间: 2014-12-09
分类: 代码人生
评论

主键

-- 为当前表添加主键
ALTER TABLE `tablename`
	ADD COLUMN id TINYINT UNSIGNED NOT NULL AUTO_INCREMENT,
	ADD PRIMARY KEY (id);

-- 删除主键

ALTER TABLE `tablename`
	DROP PRIMARY KEY;

创建数据库

# utf8mb4_unicode_ci更准
CREATE DATABASE IF NOT EXISTS typecho DEFAULT CHARSET utf8mb4 COLLATE utf8mb4_unicode_ci;
# utf8mb4_general_ci更快
CREATE DATABASE IF NOT EXISTS typecho DEFAULT CHARSET utf8mb4 COLLATE utf8mb4_general_ci;
CREATE DATABASE typecho DEFAULT CHARACTER SET gbk COLLATE gbk_chinese_ci;

创建用户并提供相应权限

# 只是创建用户
CREATE USER phpergao@'localhost' IDENTIFIED BY 'yourpasswd';

# 赋予权限
GRANT select,update on phpergao.* to phpergao@'localhost';

GRANT index ON phpergao.* TO phpergao@'192.168.0.%';

# 创建用户并赋予权限
GRANT ALL PRIVILEGES ON phpergao.* TO 'phpergao'@'localhost' IDENTIFIED BY 'yourpasswd';

# 相反的revoke 跟 grant 的语法差不多，只需要把关键字 “to” 换成 “from” 即可：
REVOKE ALL PRIVILEGES ON phpergao.* FROM 'phpergao'@'localhost';

# ALL PRIVILEGES 可以换为select,insert,update,delete,create,drop,index,alter,grant,references,reload,shutdown,process,file等14个权限。

# 删除用户
DELETE FROM user WHERE User='phpergao' and Host='localhost';

# 修改用户密码
UPDATE USER SET PASSWORD = PASSWORD ('newpasswd') WHERE	USER = 'phpergao' AND HOST = 'localhost';

刷新权限

FLUSH PRIVILEGES;

查看用户权限

# 查看自己的权限
SHOW GRANTS;
# 查看其他人的权限
SHOW GRANTS FOR 'phpergao'@'%';

新建数据表

DROP TABLE IF EXISTS `workers_info`;  
CREATE TABLE `workers_info` (
  `id` int(11) unsigned NOT NULL AUTO_INCREMENT,
  `workername` varchar(20) NOT NULL,
  `sex` enum('F','M','S') DEFAULT 'S',
  `salary` int(11) DEFAULT '0',
  `email` varchar(30) DEFAULT NULL,
  `EmployedDates` date DEFAULT NULL,
  `department` varchar(30) DEFAULT NULL,
  PRIMARY KEY (`id`)
) ENGINE=MyISAM AUTO_INCREMENT=2 DEFAULT CHARSET=utf8;

忘记了mysql密码

修改配置文件

[mysqld] 
datadir=/var/lib/mysql 
socket=/var/lib/mysql/mysql.sock 
# ADD
skip-name-resolve 
skip-grant-tables

然后重启MySQL服务并免密码登录

service mysqld restart
mysql

执行修改密码SQL

将root用户的密码统一设为'admin'
UPDATE mysql.user SET Password=password('123456') WHERE User='root';

还原MySQL配置文件并重启服务

修改用户登录HOST

UPDATE mysql.user SET Host='&' WHERE User='root';

参考：

http://renxiangzyq.iteye.com/blog/763837

File already exists: filesystem 解决办法

作者: Jimmygao
时间: 2014-12-03
分类: 代码人生
评论

今天用svn命令行提交版本的时候，碰到了这个比较麻烦的问题

svn: File already exists: filesystem 'xxx/svn/xxx/db'

搜了一下解决办法，都是需要两次commit，太麻烦。

直接在提交根目录执行以下命令

svn update path/ --accept=mine-full

一句话解决！

SVN预提交(Pre-Commit)钩子的利用

作者: Jimmygao
时间: 2014-12-02
分类: 代码人生,服务器技术
评论

转自：http://java.dzone.com/articles/useful-subversion-pre-commit

Checks whether the commit message is not empty
Checks whether the commit message consists of at least 5 characters
Checks if the committed files are UTF-8 compliant
Checks whether the svn:eol-style property is set to LF on newly added files
Checks if the committed files have no TAB characters

The UTF-8 and TAB checks are performed on the following file suffixes

*.java
*.js
*.xhtml
*.css
*.xml
*.properties (only check for TABs here, no check for UTF-8 compliance)

翻译一下：

检查提交日志是否为空
检查提交日志最少需要N个字符
检查提交文件是否是UTF-8格式
检查新文件的换行模式是否为LF
检查提交的文件是否含有TABs换行符

检查UTF-8编码和TABs换行符只针对以下后缀文件：

*.java
*.js
*.xhtml
*.css
*.xml
*.properties (只检查TABs，不检查UTF-8)

以下是代码

注意:针对Linux

#!/bin/bash

REPOS="$1"
TXN="$2"


# Make sure that the log message contains some text.
SVNLOOK=/usr/bin/svnlook
ICONV=/usr/bin/iconv

SVNLOOKOK=1
LOGMSG=`$SVNLOOK log -t "$TXN" "$REPOS" | wc -c` 
if [ "$LOGMSG" -lt 2 ];
then
   echo -e "\t That logmessage contains at least 2 alphanumeric characters. Commit aborted!" 1>&2
  exit 1
fi


# Make sure that all files to be committed are encoded in UTF-8.
while read changeline; 
do

  # Get just the file (not the add / update / etc. status).
  file=${changeline:4}

  # Only check source files.
  if [[ $file == *.java || $file == *.xhtml || $file == *.css || $file == *.xml || $file == *.js ]] ; then
    $SVNLOOK cat -t "$TXN" "$REPOS" "$file" | $ICONV -f UTF-8 -t UTF-8 -o /dev/null
    if [ "${PIPESTATUS[1]}" != 0 ] ; then
      echo "Only UTF-8 files can be committed ("$file")" 1>&2
      exit 1
    fi
  fi
done < <($SVNLOOK changed -t "$TXN" "$REPOS")

# Check files for svn:eol-style property
# Exit on all errors.
set -e
EOL_STYLE="LF"
echo "`$SVNLOOK changed -t "$TXN" "$REPOS"`" | while read REPOS_PATH
do
 if [[ $REPOS_PATH =~ A[[:blank:]]{3}(.*)\.(java|css|properties|xhtml|xml|js) ]]
 then
  if [ ${#BASH_REMATCH[*]} -ge 2 ]
    then
  FILENAME=${BASH_REMATCH[1]}.${BASH_REMATCH[2]};

  # Make sure every file has the right svn:eol-style property set
   if [ $EOL_STYLE != "`$SVNLOOK propget -t \"$TXN\" \"$REPOS\" svn:eol-style \"$FILENAME\" 2> /dev/null`" ]
    then
    ERROR=1;
      echo "svn ps svn:eol-style $EOL_STYLE \"$FILENAME\"" >&2
   fi
  fi
 fi
 test -z $ERROR || (echo "Please execute above commands to correct svn property settings. EOL Style LF must be used!" >& 2; exit 1)
done



# Block commits with tabs
# This is coded in python
# Exit on all errors
set -e

$SVNLOOK diff -t "$TXN" "$REPOS" | python /dev/fd/3 3<<'EOF'
import sys
ignore = True
SUFFIXES = [ ".java", ".css", ".xhtml", ".js", ".xml", ".properties" ]
filename = None

for ln in sys.stdin:

    if ignore and ln.startswith("+++ "):
        filename = ln[4:ln.find("\t")].strip()
        ignore = not reduce(lambda x, y: x or y, map(lambda x: filename.endswith(x), SUFFIXES))

    elif not ignore:
        if ln.startswith("+"):
        
           if ln.count("\t") > 0:
              sys.stderr.write("\n*** Transaction blocked, %s contains tab character:\n\n%s" % (filename, ln))
              sys.exit(1)

        if not (ln.startswith("@") or \
           ln.startswith("-") or \
           ln.startswith("+") or \
           ln.startswith(" ")):

           ignore = True

sys.exit(0)
EOF

# All checks passed, so allow the commit.
exit 0

如何使用

重命名hooks文件夹下pre-commit.tmpl 为 pre-commit。
修改文件内容
将文件变为可执行 chmod +x pre-commit

提交完代码自动更新

vim hooks/post-commit

#!/bin/sh
export LANG=zh_CN.UTF-8  # 设置UTF-8编码
SVN=/usr/bin/svn         # 这里配置的是svn安装bin目录下的svn文件
WEB=/var/www/html      # 要更新的目录
$SVN update $WEB --username xxxx --password xxxx

python获取操作系统平台、版本及架构

作者: Jimmygao
时间: 2014-11-19
分类: 代码人生
评论

转自: http://openwares.net/linux/python_os_version_platform.html

platform模块提供了底层系统平台的相关信息

系统架构

32位还是64位

>>> import platform
>>> platform.architecture()
('64bit', 'ELF') # python 3.3.2+ 64 bits on debian jessie 64 bits
('32bit', 'WindowsPE') # python 3.3.2 32 bits on windows 8.1 64 bits
('64bit', 'WindowsPE') # python 3.3.2 64 bits on wndows 8.1 64 bits
('64bit', '') # python 3.4.1 64 bits on mac os x 10.9.4
ELF和WindowsPE是可执行文件格式

操作系统

linux,mac还是windows

>>> platform.system()
'Linux' # python 3.3.2+ 64 bits on debian jessie 64 bits
'Windows' # python 3.3.2 32 bits on windows 8.1 64 bits
'Windows' # python 3.3.2 64 bits on windows 8.1 64 bits
'Darwin' # python 3.4.1 64 bits on mac os x 10.9.4

系统版本

>>> platform.version()
'#1 SMP Debian 3.10.11-1 (2013-09-10)' # python 3.3.2+ 64 bits on debian jessie 64 bits
'6.2.9200' # python 3.3.2 32 bits on windows 8.1 64 bits
'6.2.9200' # python 3.3.2 64 bits on windows 8.1 64 bits
'Darwin Kernel Version 13.3.0: Tue Jun  3 21:27:35 PDT 2014; root:xnu-2422.110.17~1/RELEASE_X86_64' # python 3.4.1 64 bits on mac os x 10.9.4

CPU平台

>>> platform.machine()
'x86_64' # python 3.3.2+ 64 bits on debian jessie 64 bits
'AMD64' # python 3.3.2 32 bits on windows 8.1 64 bits
'AMD64' # python 3.3.2 64 bits on windows 8.1 64 bits
'x86_64' # python 3.4.1 64 bits on mac os x 10.9.4

linux发行版

>>> platform.dist()
('debian', 'jessie/sid', '') # python 3.3.2+ 64 bits on debian jessie 64 bits

节点名

也就是机器名

>>> platform.node()
'work' # python 3.3.2+ 64 bits on debian jessie 64 bits
'work-xxx' # python 3.3.2 32 bits on windows 8.1 64 bits

系统信息

>>> platform.uname()
uname_result(system='Linux', node='work', release='3.10-3-amd64', version='#1 SMP Debian 3.10.11-1 (2013-09-10)', machine='x86_64', processor='') # python 3.3.2+ 64 bits on debian jessie 64 bits
 
uname_result(system='Windows', node='work-xxx', release='8', version='6.2.9200', machine='AMD64', processor='Intel64 Family 6 Model 58 Stepping 9,
GenuineIntel') # python 3.3.2 32 bits on windows 8.1 64 bits
 
uname_result(system='Darwin', node='mba', release='13.3.0', version='Darwin Kernel Version 13.3.0: Tue Jun  3 21:27:35 PDT 2014; root:xnu-2422.110.17~1/RELEASE_X86_64', machine='x86_64', processor='i386') # python 3.4.1 64 bits on mac os x 10.9.4

python版本

>>> platform.python_verison()
'3.3.2+' # python 3.3.2+ 64 bits on debian jessie 64 bits
'3.3.3' # python 3.3.2 32 bits on windows 8.1 64 bits