Archive for the ‘Uncategorized’ Category
Interesting links
Kinder egg banned in the US? Really, wtf USA.
http://www.eatliver.com/i.php?n=7410
http://news.blogs.cnn.com/2011/04/21/easter-reminder-kinder-eggs-banned-in-the-united-states/
Vancouver riots at loss
http://www.eatliver.com/i.php?n=7413
Making Love, Not War http://yfrog.com/z/h03elanp
I’m sure someone already is working on a “Will it Fry?” blog
http://www.wimp.com/friedkoolaid/
Now that’s fast, even though i’m not sure it hit 500km/h in this video..
http://www.wimp.com/maglevtrain/
I would do this in a heart beat. Although I’d be scared, the lion would sense that and likely bite
my head off.
http://www.wimp.com/lionnap/
What Should I Do? Choosing SQL, NoSQL or Both
how to use a phone
cholula, honey and rice
I was too lazy to make any real food and had some left over rice. Give it a try, it’s quite amazing.
Excavator lifting itself to the top
future olympic event
drop dead simple HOME synchronization
Something that is always annoying to me is the setup, scripting and management of keeping a hosts files and scripts in a safe place. Well i’ve made various reincarnations of various scripts that attempt to do this. I’ll ommit the history here but wanted to drop a quick note of a drop dead simple solution that has worked pretty well for me.
drop the contents below into Makefile and git init it.
.PHONY: backup all: @echo try 'make backup' or 'make restore' backup: @echo backup file from system started rsync -arv --files-from manifest.txt / backup/ python filemeta.py backup -P backup/ restore: @echo restore file to system started rsync -arv --files-from manifest.txt backup/ / python filemeta.py restore
Since git does not preserve file mode and permission I hacked up a quick script to handle backup and restore of the key file meta data (mode and ownership). Save the contents here as filemeta.py
import sys, os, stat
from optparse import OptionParser
_, cmd = sys.argv.pop(0), sys.argv.pop(0)
parser = OptionParser()
parser.add_option("-c", "--config", dest="config", help="config file", metavar="FILE", default=None)
parser.add_option("-v", "--verbose", dest="verbose", action="store_true", default=False)
parser.add_option("-f", "--file-meta", dest="file_meta", default="./file.meta", help="store file meta data path [%default]")
parser.add_option("-P", "--path", dest="path", default=".", help="path to scan [%default]")
if cmd == 'backup':
(options, args) = parser.parse_args(sys.argv)
path = os.path.realpath(options.path)
file_meta = os.path.realpath(options.file_meta)
file_meta_tmp = file_meta + ".tmp"
fh = open(file_meta_tmp, "w+")
for dirpath, dirnames, filenames in os.walk(path):
st = os.stat(dirpath)
fh.write("%d %d %d %s\n" % (st.st_mode, st.st_uid, st.st_gid, dirpath))
for filename in filenames:
fullpath = os.path.join(dirpath, filename)
st = os.stat(fullpath)
abspath = fullpath.replace(path, '')
fh.write("%d %d %d %s\n" % (st.st_mode, st.st_uid, st.st_gid, abspath))
fh.close()
os.rename(file_meta_tmp, file_meta)
print "Created", file_meta
elif cmd == 'restore':
(options, args) = parser.parse_args(sys.argv)
path = os.path.realpath(options.path)
file_meta = os.path.realpath(options.file_meta)
for line in file(file_meta):
st_mode, st_uid, st_gid, abspath = line.split(None, 3)
st_mode, st_uid, st_gid = int(st_mode), int(st_uid), int(st_gid)
abspath = abspath.strip()
if stat.S_ISDIR(st_mode) or stat.S_ISREG(st_mode):
os.chmod(abspath, st_mode)
os.chown(abspath, st_uid, st_gid)
else:
continue
else:
raise Exception("invalid command: %s" % (cmd))
Now all you have to do is place the filenames to backup (files or directories) in manifest.txt and run ‘sudo make backup’ or ‘sudo make restore’. git commit or add backup/* and push it off to a origin of your choice. This is nothing that complex but the simple approach to this has made box recovery and migration a breeze. Lots of times you want to track something but don’t have a good place to put it. This is that place. all of those pesky $HOME/.bash_profile files are now in one place and can easily be fetched from a single location. I’ll leave branching and cherry picking to the reader.
Acp
Acp.
– SNIP –
- Directory data
- Inode data (things returned by stat(2))
- File bodies
Acp creates queues corresponding to each of these groups, and tries to do work in bulk in each one. As it finds files and directories the are sorted by either inode number (the default) or by the first block in the file (acp -b). Run acp -h to see the other command line options.
In general, the sorting makes the IO started by acp much closer to the order the filesystems store things on disk. It is especially effective on ext3 when htree is used, because htree causes readdir to return files in random order. This graph shows the performance difference while reading a single kernel tree with tar and with acp.
– SNIP –
brick
Varnish, Couch and S3
Sending screenshots around is something I do quite often. Rather than rely on external screenshot services I like to build my own using available cloud services. Because doing everything on the cheap is better, more fun and challenging.
Configuring varnish to fetch assets from CouchDB and then S3 as a fail safe if couch is MIA was a fun use of available technologies. Couch lets you save attachments in documents and it was quite a simple addition to the screenshot program I use.
python screenshot program
# add metadata record and image to couchdb
couchone_uri = "http://XXXXXX.couchone.com/screenshot"
scheme, netloc, path, query, fragment = urlparse.urlsplit(couchone_uri)
ss = httplib.HTTPConnection(netloc)
# save meta data
rec = {
"year" : int(time.strftime("%Y")),
"month" : int(time.strftime("%m")),
"day" : int(time.strftime("%d")),
"doctype" : "metadata",
"s3_url" : s3_url,
"s3_key" : s3_key,
}
headers = {
"Content-Type" : "application/json",
}
couch_key = s3_key.replace("/", "-")
ss.request("PUT", os.path.join(path, couch_key), json.dumps(rec), headers)
res = ss.getresponse()
ret = json.loads(res.read())
print res.status, res.reason, ret
# Upload the image as an attachment
aurl = os.path.join(path, ret['id'], "image")
aurl_params = "?rev=" + ret['rev']
ss.request("PUT", aurl + aurl_params, file(filename).read(), { "Content-Type" : "image/jpg" })
res = ss.getresponse()
print res.status, res.reason, res.read()
attach_url = scheme + "://" + netloc + os.path.join(path , ret['id'], "image")
print "URL", repr(attach_url)


