SQL Developer Keepalive plugin

When you start SQL Developer, click on the icon.  Pressing enter gives you the default of sending a message every 600 seconds.

https://github.com/scristalli/SQL-Developer-4-keepalive

blogodex = {"idx" : {"SQL Developer", "Oracle connection timeout", "ORA-03135"]};

What if your name is actually Null?

Interesting comments by Mr. Christopher Null!

I'm adding "null" to the blogodex, will it come back to bite me?

http://www.wired.com/2015/11/null

blogodex = {"idx" : ["problem areas", "null"]};

Nice article on The Log

"Each working data pipeline is designed like a log; each broken data pipeline is broken in its own way."—Count Leo Tolstoy (translation by Jay Kreps)

Here's an excellent manifesto about "the data log".  You might be familiar with this concept in the guise of database journals, or with event sequences in distributed systems.

Jay starts from there and writes (pretty comprehensively!) about how this idea is used in several modern systems, and talks about his experience at linkedin using this approach.  Lots of interesting references at the end.  I was happy to see John Ousterhout mentioned there!

The Log: What every software engineer should know about real-time data's unifying abstraction

blogodex = {"idx" : ["logging", "Jay Kreps", "Kafka", "fundamentals", "scalability"]};

Some JVM Resources

Three interesting sources:

• JVM Bytecode for Dummies
• Create Your Own Programming Language
• Implement Your Own JVM Compiler

JVM Bytecode for Dummies

Nice presentation by Charles Nutter.

http://www.slideshare.net/CharlesNutter/redev-2010-jvm-bytecode-for-dummies

Monitoring Oracle Dataguard Status

Received a tip from the gurus:

ON STANDBY:
v$dataguard_stats - various stats, including apply lag.
v$dataguard_status - status messages related to dataguard.
v$managed_standby - info on the standby 

ON PRIMARY
v$archive_gap - shows the # of archived redo logs the standby is
behind.  Since we're shipping in real-time, this should nearly always be 0.

Boto 3 Released

Key points:

• works with python 2 and 3
• interoperable with boto
• new package name: boto3
• docs here: https://boto3.readthedocs.org/
• pip install boto3

https://aws.amazon.com/blogs/aws/now-available-aws-sdk-for-python-3-boto3/

Setting a Sphinx theme

To set a theme in Sphinx:

• find a theme you want to copy.  It will generally be a directory full of files.
• copy the directory under your project's _templates directory.
• set the html_theme in conf.py
• some docs mention setting html_theme_path as well, but it doesn't seem necessary if it's under _templates.
• There's some nice themes at https://readthedocs.org

Example:

$ ls _templates/

sphinx_rtd_theme/

$ diff conf.py-orig conf.py

113c113,114

< html_theme = 'alabaster'

---

> html_theme = "sphinx_rtd_theme"

> html_theme_path = ["_themes", ]    ### probably not necessary

"git push" to Github wants me to log in!

If git prompts you for a username when you push:

$ git push

Username: 

Check your remote settings.

$ git remote -v

origin https://github.com/YOURNAME/notebook.git (fetch)

origin https://github.com/YOURNAME/notebook.git (push)

If it's an https reference, you can change it to an ssh reference as follows. 

$ git remote set-url origin git@github.com:YOURNAME/notebook.git

And everything is great!

$ git push

Everything up-to-date

Mathematical Theory of Claude Shannon

Claude Shannon is well-known for having pioneered information theory and switching theory.  Here's a great overview of his approach to these and other areas he studied.

http://web.mit.edu/6.933/www/Fall2001/Shannon1.pdf

Software: Pagegen Static Site Generator

Pagegen converts a directory tree of RST docs into a static site.  "Aims to generate a fully featured site ready for CSS styling," according to the docs.

blogodex = {"toc" : "PageGen", "idx" = ["Software"]};

Book: The Three Body Problem, Liu Cixin

When I was in China, I met some Sci Fi enthusiasts and asked who their favorite authors were.  I wasn't sure who might have been translate, and how the various political points of view might have affected what was brought in to the country.

To my surprise, they had a lot of authors, all of them Chinese.  It turns out that the country's enthusiasm for science led to Science Fiction being very popular in the country.

I hadn't heard of any of them, and none of the works seem to have been translated into English. Perhaps this is going to change, starting with this book?

Amazon link and their blurb:

The Three-Body Problem is the first chance for English-speaking readers to experience this multiple award winning phenomenon from China's most beloved science fiction author, Liu Cixin. 

Set against the backdrop of China's Cultural Revolution, a secret military project sends signals into space to establish contact with aliens. An alien civilization on the brink of destruction captures the signal and plans to invade Earth. Meanwhile, on Earth, different camps start forming, planning to either welcome the superior beings and help them take over a world seen as corrupt, or to fight against the invasion. The result is a science fiction masterpiece of enormous scope and vision.

blogodex = {"idx" : ["Books", "Science Fiction", "Three Body Problem", "Liu Cixin"]};

Requests Notes: Python HTTP API

A very pleasant HTTP library for Python.

"Requests takes all of the work out of Python HTTP/1.1 — making your integration with web services seamless. There’s no need to manually add query strings to your URLs, or to form-encode your POST data. Keep-alive and HTTP connection pooling are 100% automatic, powered by urllib3, which is embedded within Requests."

References

• Main Page, with good docs
• "SSL InsecurePlatform Error"

Incantations


blogodex = {"toc" : "requests", "idx" = ["python", "http"]};

Good post on testing server failures

tl;dr: make your system "pressure tolerant" by using system tools to simulate failures.

If there's one thing to know about distributed systems, it's that they have to be designed with the expectation of failure. It's also safe to say that most software these days is, in some form, distributed—whether it's a database, mobile app, or enterprise SaaS. If you have two different processes talking to each other, you have a distributed system, and it doesn't matter if those processes are local or intergalactically displaced. 

Marc Hedlund recently had a great post on Stripe's game-day exercises where they block off an afternoon, take a blunt instrument to their servers, and see what happens. We're talking like abruptly killing instances here—kill -9, ec2-terminate-instances, yanking on the damn power cord—that sort of thing. Everyone should be doing this type of stuff. You really don't know how your system behaves until you see it under failure conditions.

http://www.bravenewgeek.com/sometimes-kill-9-isnt-enough/

Two Tcl Oldies...

I was surprised to see these bits of fiddlefaddle online:

The Tao of Tcl
A Class Code Generator

Building RESTful APIs with Tornado

Nice Tornado Article.

http://www.drdobbs.com/open-source/building-restful-apis-with-tornado/240160382

from datetime import date

import tornado.escape

import tornado.ioloop

import tornado.web

 

class VersionHandler(tornado.web.RequestHandler):

    def get(self):

        response = { 'version': '3.5.1',

                     'last_build':  date.today().isoformat() }

        self.write(response)

 

class GetGameByIdHandler(tornado.web.RequestHandler):

    def get(self, id):

        response = { 'id': int(id),

                     'name': 'Crazy Game',

                     'release_date': date.today().isoformat() }

        self.write(response)

 

application = tornado.web.Application([

    (r"/getgamebyid/([0-9]+)", GetGameByIdHandler),

    (r"/version", VersionHandler)

])

 

if __name__ == "__main__":

    application.listen(8888)

    tornado.ioloop.IOLoop.instance().start()

The Night Watch

One of the most awesome computer papers ever written!

http://research.microsoft.com/en-us/people/mickens/thenightwatch.pdf

When you debug a distributed system or an OS kernel, you do it Texas-style. You gather some mean, stoic people, people who have seen things die, and you get some primitive tools, like a compass and a rucksack and a stick that’s pointed on one end, and you walk into the wilderness and you look for trouble, possibly while using chewing tobacco. As a systems hacker, you must be prepared to do savage things, unspeakable things, to kill runaway threads with your bare hands, to write directly to network ports using telnet and an old copy of an RFC that you found in the Vatican. When you debug systems code, there are no high-level debates about font choices and the best kind of turquoise, because this is the Old Testament, an angry and monochromatic world, and it doesn’t matter whether your Arial is Bold or Condensed when people are covered in boils and pestilence and Egyptian pharaoh oppression. HCI people discover bugs by receiving a concerned email from their therapist. Systems people discover bugs by waking up and discovering that their first-born children are missing and “ETIMEDOUT” has been written in blood on the wall.

Mechanical Soup: A Python library for automating interaction with websites

MechanicalSoup automatically stores and sends cookies, follows redirects, and can follow links and submit forms. It doesn't do Javascript.

https://github.com/hickford/MechanicalSoup

Linux: Invalidating Cache

Here's the incantation for invalidating the linux file system cache.  Good for running performance tests when you don't want a hot cache throwing off the results.

sudo sh -c 'sync;echo 3 > /proc/sys/vm/drop_caches' 

Pymunk

"pymunk is a easy-to-use pythonic 2d physics library that can be used whenever you need 2d rigid body physics from Python."

https://github.com/viblo/pymunk