Mein Neffe auf den Philippinen

Heute mal ein Blogpost in ganz eigener Sache.
Und zwar macht mein Neffe Steve gerade seinen Zivildienst / freiwilliges Jahr auf den Philippinen. Er ist dort im Rahmen einer Organisationen mit dem Namen PREDA und dem Tatort Straßen der Welt e.V, und betreut dort Kinder, welche von der Organisation aus Gefaengnissen geholt werden. Genaueres kann man auf seinem Blog nachlesen, wo er heute schon einen ersten Beitrag kurz nach seiner Ankunft geschrieben hat und erste Bilder hochgeladen hat.

Doch auf eine wichtige Sache moechte ich noch hinweisen:
Steve, seine Mitstreiter und die Organisation sind in diesem Projekt auf Spenden angewiesen. D.h. wenn ihr gut findet, was Steve dort macht, dann koennt ihr dies mit einer Spende unterstuetzen. Steve moechte auf seiner Website dann dokumentieren, wofuer die Spenden eingesetzt werden. Die Organisation kann ab 50 EUR auch Spendenquittungen austellen.

Details zum Spenden findet ihr hier.

Veröffentlicht unter Allgemein | Verschlagwortet mit , | Hinterlasse einen Kommentar

11 links on scalable software architecture and big data

From time to time I go through my bookmarks and fetch interesting links I came across during the last couple of month. This article is kind of Part 2 of a past article with links and videos about scalability and software architecture.

1. Learning from Five Years as a Skype Architect – Andres Kutt
2. Scale at Facebook
3. Murder: Fast datacenter code deploys using BitTorrent
4. What every programmer should know about memory, Part 1
5. Multithreading Magic
6. NoSQL  The Definitive Guide
7. Getting Good IO from Amazon’s EBS
8. Building a terabyte-scale data cycle at LinkedIn with Hadoop and Project Voldemort
9. Scalability of the Hadoop Distributed File System
10. Understanding Cassandra Code Base
11. Big Data in Real-Time at Twitter

Veröffentlicht unter Software-Development | Verschlagwortet mit , , , , | Hinterlasse einen Kommentar

How to create a valid SAML 2.0 Assertion with OpenSAML for JAVA

Synesty: Automation – API Connectors – Datafeed Management – Without programming


Recently I had to implement SAML 2.0 for a client project for a Single Sign On (SSO) process.
For this purpose our system needed to create a SAML Assertion containing all the information of the user who is currently logging in.

I was using the OpenSAML classes for JAVA.
Initially I had some issues because I couldn’t find good documentation / examples about what how such an Assertion is to be created using OpenSAML.

I had to do lots of googling and trial & error and finally I got it working.
The following sites were useful and led to my solution.

Though I had to make modifications to some of the solutions above.

To save you this pain, I am posting my final example here. The class below is only for demonstration purposes of this blog entry, and I have stripped out lots of other stuff (e.g. error handling, client specific stuff, encryption via EncryptedAssertion). The only purpose is to give you some kind of template and idea how to start.

import java.util.HashMap;
import java.util.Map;
import java.util.Set;
import org.joda.time.DateTime;
import org.opensaml.Configuration;
import org.opensaml.DefaultBootstrap;
import org.opensaml.common.SAMLObjectBuilder;
import org.opensaml.common.SAMLVersion;
import org.opensaml.saml2.core.Assertion;
import org.opensaml.saml2.core.Attribute;
import org.opensaml.saml2.core.AttributeStatement;
import org.opensaml.saml2.core.AttributeValue;
import org.opensaml.saml2.core.AuthnContext;
import org.opensaml.saml2.core.AuthnContextClassRef;
import org.opensaml.saml2.core.AuthnStatement;
import org.opensaml.saml2.core.Condition;
import org.opensaml.saml2.core.Conditions;
import org.opensaml.saml2.core.Issuer;
import org.opensaml.saml2.core.NameID;
import org.opensaml.saml2.core.OneTimeUse;
import org.opensaml.saml2.core.Subject;
import org.opensaml.saml2.core.SubjectConfirmation;
import org.opensaml.saml2.core.SubjectConfirmationData;
import org.opensaml.saml2.core.impl.AssertionMarshaller;
import org.opensaml.xml.ConfigurationException;
import org.opensaml.xml.XMLObjectBuilder;
import org.opensaml.xml.XMLObjectBuilderFactory;
import org.opensaml.xml.schema.XSString;
import org.opensaml.xml.util.XMLHelper;
import org.w3c.dom.Element;
 * This is a demo class which creates a valid SAML 2.0 Assertion.
public class SAMLWriter
	public static void main(String[] args) {
    	try {
			SAMLInputContainer input = new SAMLInputContainer();
			input.strIssuer = "";
			input.strNameID = "UserJohnSmith";
			input.strNameQualifier = "My Website";
			input.sessionId = "abcdedf1234567";
			Map customAttributes = new HashMap();
			customAttributes.put("FirstName", "John");
			customAttributes.put("LastName", "Smith");
			input.attributes = customAttributes;
			Assertion assertion = SAMLWriter.buildDefaultAssertion(input);
			AssertionMarshaller marshaller = new AssertionMarshaller();
			Element plaintextElement = marshaller.marshall(assertion);
			String originalAssertionString = XMLHelper.nodeToString(plaintextElement);
			System.out.println("Assertion String: " + originalAssertionString);
			// TODO: now you can also add encryption....
		} catch (MarshallingException e) {
			// TODO Auto-generated catch block
	private static XMLObjectBuilderFactory builderFactory;
	public static XMLObjectBuilderFactory getSAMLBuilder() throws ConfigurationException{
		if(builderFactory == null){
			// OpenSAML 2.3
	         builderFactory = Configuration.getBuilderFactory();
		return builderFactory;
	 * Builds a SAML Attribute of type String
	 * @param name
	 * @param value
	 * @param builderFactory
	 * @return
	 * @throws ConfigurationException
	public static Attribute buildStringAttribute(String name, String value, XMLObjectBuilderFactory builderFactory) throws ConfigurationException
		SAMLObjectBuilder attrBuilder = (SAMLObjectBuilder) getSAMLBuilder().getBuilder(Attribute.DEFAULT_ELEMENT_NAME);
		 Attribute attrFirstName = (Attribute) attrBuilder.buildObject();
		 // Set custom Attributes
		 XMLObjectBuilder stringBuilder = getSAMLBuilder().getBuilder(XSString.TYPE_NAME);
		 XSString attrValueFirstName = (XSString) stringBuilder.buildObject(AttributeValue.DEFAULT_ELEMENT_NAME, XSString.TYPE_NAME);
		return attrFirstName;
	 * Helper method which includes some basic SAML fields which are part of almost every SAML Assertion.
	 * @param input
	 * @return
	public static Assertion buildDefaultAssertion(SAMLInputContainer input)
	         // Create the NameIdentifier
	         SAMLObjectBuilder nameIdBuilder = (SAMLObjectBuilder) SAMLWriter.getSAMLBuilder().getBuilder(NameID.DEFAULT_ELEMENT_NAME);
	         NameID nameId = (NameID) nameIdBuilder.buildObject();
	         // Create the SubjectConfirmation
	         SAMLObjectBuilder confirmationMethodBuilder = (SAMLObjectBuilder)  SAMLWriter.getSAMLBuilder().getBuilder(SubjectConfirmationData.DEFAULT_ELEMENT_NAME);
	         SubjectConfirmationData confirmationMethod = (SubjectConfirmationData) confirmationMethodBuilder.buildObject();
	         DateTime now = new DateTime();
	         SAMLObjectBuilder subjectConfirmationBuilder = (SAMLObjectBuilder) SAMLWriter.getSAMLBuilder().getBuilder(SubjectConfirmation.DEFAULT_ELEMENT_NAME);
	         SubjectConfirmation subjectConfirmation = (SubjectConfirmation) subjectConfirmationBuilder.buildObject();
	         // Create the Subject
	         SAMLObjectBuilder subjectBuilder = (SAMLObjectBuilder) SAMLWriter.getSAMLBuilder().getBuilder(Subject.DEFAULT_ELEMENT_NAME);
	         Subject subject = (Subject) subjectBuilder.buildObject();
	         // Create Authentication Statement
	         SAMLObjectBuilder authStatementBuilder = (SAMLObjectBuilder) SAMLWriter.getSAMLBuilder().getBuilder(AuthnStatement.DEFAULT_ELEMENT_NAME);
	         AuthnStatement authnStatement = (AuthnStatement) authStatementBuilder.buildObject();
	         DateTime now2 = new DateTime();
	         SAMLObjectBuilder authContextBuilder = (SAMLObjectBuilder) SAMLWriter.getSAMLBuilder().getBuilder(AuthnContext.DEFAULT_ELEMENT_NAME);
	         AuthnContext authnContext = (AuthnContext) authContextBuilder.buildObject();
	         SAMLObjectBuilder authContextClassRefBuilder = (SAMLObjectBuilder) SAMLWriter.getSAMLBuilder().getBuilder(AuthnContextClassRef.DEFAULT_ELEMENT_NAME);
	         AuthnContextClassRef authnContextClassRef = (AuthnContextClassRef) authContextClassRefBuilder.buildObject();
	         authnContextClassRef.setAuthnContextClassRef("urn:oasis:names:tc:SAML:2.0:ac:classes:Password"); // TODO not sure exactly about this
	        // Builder Attributes
	         SAMLObjectBuilder attrStatementBuilder = (SAMLObjectBuilder) SAMLWriter.getSAMLBuilder().getBuilder(AttributeStatement.DEFAULT_ELEMENT_NAME);
	         AttributeStatement attrStatement = (AttributeStatement) attrStatementBuilder.buildObject();
	      // Create the attribute statement
	         Map attributes = input.getAttributes();
	         if(attributes != null){
	        	 Set keySet = attributes.keySet();
	        	 for (String key : keySet)
	        		 Attribute attrFirstName = buildStringAttribute(key, attributes.get(key), getSAMLBuilder());
	         // Create the do-not-cache condition
	         SAMLObjectBuilder doNotCacheConditionBuilder = (SAMLObjectBuilder) SAMLWriter.getSAMLBuilder().getBuilder(OneTimeUse.DEFAULT_ELEMENT_NAME);
	         Condition condition = (Condition) doNotCacheConditionBuilder.buildObject();
	         SAMLObjectBuilder conditionsBuilder = (SAMLObjectBuilder) SAMLWriter.getSAMLBuilder().getBuilder(Conditions.DEFAULT_ELEMENT_NAME);
	         Conditions conditions = (Conditions) conditionsBuilder.buildObject();
	         // Create Issuer
	         SAMLObjectBuilder issuerBuilder = (SAMLObjectBuilder) SAMLWriter.getSAMLBuilder().getBuilder(Issuer.DEFAULT_ELEMENT_NAME);
	         Issuer issuer = (Issuer) issuerBuilder.buildObject();
	         // Create the assertion
	         SAMLObjectBuilder assertionBuilder = (SAMLObjectBuilder) SAMLWriter.getSAMLBuilder().getBuilder(Assertion.DEFAULT_ELEMENT_NAME);
	         Assertion assertion = (Assertion) assertionBuilder.buildObject();
			return assertion;
		catch (Exception e)
		return null;
	public static class SAMLInputContainer
		private String strIssuer;
		private String strNameID;
		private String strNameQualifier;
		private String sessionId;
		private int maxSessionTimeoutInMinutes = 15; // default is 15 minutes
		private Map attributes;
		 * Returns the strIssuer.
		 * @return the strIssuer
		public String getStrIssuer()
			return strIssuer;
		 * Sets the strIssuer.
		 * @param strIssuer
		 *            the strIssuer to set
		public void setStrIssuer(String strIssuer)
			this.strIssuer = strIssuer;
		 * Returns the strNameID.
		 * @return the strNameID
		public String getStrNameID()
			return strNameID;
		 * Sets the strNameID.
		 * @param strNameID
		 *            the strNameID to set
		public void setStrNameID(String strNameID)
			this.strNameID = strNameID;
		 * Returns the strNameQualifier.
		 * @return the strNameQualifier
		public String getStrNameQualifier()
			return strNameQualifier;
		 * Sets the strNameQualifier.
		 * @param strNameQualifier
		 *            the strNameQualifier to set
		public void setStrNameQualifier(String strNameQualifier)
			this.strNameQualifier = strNameQualifier;
		 * Sets the attributes.
		 * @param attributes
		 *            the attributes to set
		public void setAttributes(Map attributes)
			this.attributes = attributes;
		 * Returns the attributes.
		 * @return the attributes
		public Map getAttributes()
			return attributes;
		 * Sets the sessionId.
		 * @param sessionId the sessionId to set
		public void setSessionId(String sessionId)
			this.sessionId = sessionId;
		 * Returns the sessionId.
		 * @return the sessionId
		public String getSessionId()
			return sessionId;
		 * Sets the maxSessionTimeoutInMinutes.
		 * @param maxSessionTimeoutInMinutes the maxSessionTimeoutInMinutes to set
		public void setMaxSessionTimeoutInMinutes(int maxSessionTimeoutInMinutes)
			this.maxSessionTimeoutInMinutes = maxSessionTimeoutInMinutes;
		 * Returns the maxSessionTimeoutInMinutes.
		 * @return the maxSessionTimeoutInMinutes
		public int getMaxSessionTimeoutInMinutes()
			return maxSessionTimeoutInMinutes;
Veröffentlicht unter Software-Development | Verschlagwortet mit , , , , | 499 Kommentare

HTML Snippet: IE specific CSS classes for body tag

A colleque just showed me today how you can give the body tag a specific CSS class depending on the versions of Internet Explorer. With this you can avoid special css files for IE specific styling and handle it inside a single CSS file.
A colleque just showed me today how you can give the body tag a specific CSS class depending on the versions of Internet Explorer. With this you can avoid special css files for IE specific styling and handle it inside a single CSS file.

Veröffentlicht unter Software-Development | 58 Kommentare

My Notes taken at Berlin Buzzwords


Keynote – Grant Ingersoll – Lucid Imagination about Lucene

  • @gsingers
  • SolrCloud == ZooKeeper + Solr
  • Map/Reduce Ready recommenders available
  • Identify Topics
  • Latent Dirichlet Allocation ==>
  • Frequent Pattern Mining
  • Clustering
  • K-Means, Dirichlet, Canopy, etc
  • Carrot^2 Document and Search Result Clustering =>
  • Eigent Cuts (spectral Clustering) =>
  • Location Aware Search results
  • Query Parsing
  • Filtering
  • Boosting
  • Sorting
  • Singular Value Decomposition (SVD) =>
  • technique for reducing the dimensionaltity of large matrices while retaining the core features of the larger space
  • Latent Semantic Analysis uses SVD to provide search over the reduced space =>
  • Named Entity Recognition =>
  • Finite-State Queries in Lucene

Lucene Forecast – Version, Unicode, Flex, Modules by Simon Willnauer

  • switched to Kino 10

Making Software for Humans: CouchDB

  • Jan Lehnardt
  • a one-size-fits-all solution for scaling out is not really possible at all @janl #berlinbuzzwords

“Text and Metadata extraction with Apache Tika

  • switched to Riak Talk




  • best talk so far

Elastic Search

  • distributed
  • completly HTTP based
  • range queries possible
  • json based
  • lucene based
  • filters: faster than queries (cachable)
  • Near Realtime Search available

    NeoJ – Peter Neubauer

  • Nodes
  • Relationsships between nodes
  • properties on Nodes and props
  • Traversal Framework
  • Lucene integrated (indexing done on commit)

Riak Search

  • different query type patterns – keybased
  • Consistent Hashing and Partitions
  • Optimizations (to avoid “Obama Problem = Hotspots in the ring e.g. many documents containing the word Obama all going to the same node”)
  • Bloom Filters & Caching
  • Batching to sae query-time & index-time bandwith



  • ZeroMQ – 0MQ –
  • “the fewest possible moving pieces” -> no service discovery.
  • No Broker

Hypertable –

  • used by Baidu search engine
  • Dynamo – used by Amazon for Shopping Cart (uses Read-Repair and Consistent Hashing)
  • during periods of failure there can be latency spikes, because of single machine handling the ranges / machines
  • LSM – Log Structured Merge Tre
  • eleminates random I/O => holding a log tree strucutre in memory which will be written to disk asynchronously (compaction)
  • uses Bloom Filters => helps to avoid disk seeks by running every key through bloom filter to determine in which file the k/v is in
  • Dynamic Memeory Adjustment based on Workload
  • Hypertable vs. HBase
  • 70% faster than hBase (seq. write / seq. read)
  • one of the reasons: Dynamic Memory Adjustment
  • You can run Map/Reduce (Hadoop) and it is Data Locality Aware
  • Query types: timestamp ranges, versions of the cell possible, but mainly primary key access
  • No delete operation currenlty (but each column family has a TTL)

HDFS Deep Dive

  • one name node, multiple data nodes
  • use 0.20.2-append branch
  • NameNode SPOF in theory, but not in practise

Cassandra Talk – Eric Evans

  • Vectorclocks =>

Massivly Parallel Analytics beyond Map/Reduce

  • Context: StratoSphere Project

Colaborative Filtering Mahout – Frank Scholten

  • is a machine learning lib for java
  • run on hadoop
  • Similarity Algorithms
  • TanimotoCoefficintSimilarity
  • LoglikelihoodSimilarity

Sqoop Database import/export for hadoop

  • wish every speaker would have had logically structured their talks as Aaron Kimball did with his Hadoop talks


  • parser/optimizer/compiler that translates HiveQL into MapReduce code
  • metastore that stores “schema” information (e.g. table name, column names, data types)
  • schema on read, not write
  • unbedingt anschauen!!!
  • WHERE => map
  • GROUP BY/ORDER BY => reduce
  • JOIN => map or reduce depending on optimizer
  • there is an HiveQL EXPLAIN command
  • python lernen
Veröffentlicht unter Software-Development | Verschlagwortet mit , , , | 3 Kommentare

Gefüllte Paprikaschoten und geiler Wein

Am Wochenende stand ein unter Freunden lang erwartetes Kochevent ins Haus, was gleich mal Anlass fuer eine neue Kategorie auf der Keksrolle ist.

Nach einigen Ueberlegungen fiel die Wahl auf folgendes Rezept„Gefüllte Paprika nach Uroma Susanne“. Das ganze ist vor allem zu empfehlen, da man eben die Paprikaschoten sehr exakt portionieren kann. Eine grosse Paprikaschote pro Person reicht absolut aus. Dazu noch etwas Wildreis gekocht und fertig war die Mahlzeit fuer 8 Personen.

Als Vorspeise gab es „Tortilla Schnecken“, welche aber nicht von mir zubereitet wurden und wir auch leicht abgewandelt haben.

Und ein Dessert darf natürlich auch nicht fehlen. Es gab handgemachte Erdbeer- und Heidelbeertörtchen.

Abgerundet wurde unsere Feierlichkeit von diversen Flaschen „geilem“ Wein, vom Weingut Geil, von denen ich mir neulich eine Kiste des 2009er Jahrgangs bestellt hatte. Wir hatten zwar noch viele weitere Weine im Haus, allerdings wollten wir nach der ersten Flasche Grauburgunder sowie einer Flasche Spätburgunder Rosé nicht mehr wechseln. Es folgten darauf noch ein Gewürztraminer sowie ein Riesling aus der Selection Reihe.

Weingut Geil Eimsheim - Grauburgunder, Spaetburgunder, Silvaner, Gewuerztraminer, Riesling trocken

Fazit: Ein toller Abend mit Freunden, leckerem Essen, viel Wein und spielen 🙂

Veröffentlicht unter Kochen | Verschlagwortet mit | 181 Kommentare

I’m going to Berlin Buzzwords

On June 7th and 8th, I am going to Berlin to attend the Berlin Buzzwords conference.

This is what they write about themselves:

Berlin Buzzwords 2010 is a conference for developers and users of open source software projects, focussing on the issues of scalable search, data-analysis in the cloud and NoSQL-databases. Berlin Buzzwords presents more than 30 talks and presentations of international speakers specific to the three tags „search“, „store“ and „scale“.

I'm going to Berlin Buzzwords - the conference on searching, processing and storing data.

Veröffentlicht unter Software-Development | 1 Kommentar

„real time“ still hasn’t made it into corporate IT

today, when I requested a password reset for my account (I forgot the password)

oh you want this to work immediately ???…:) sorry [name removed] you have to wait 45 min for the account to propogate through the network….not my fault there….just the way things work here at [name removed] …:)
and sorry I can’t make it work any faster just the way the network is setup

this proves that „real time“ still hasn’t made it into corporate IT….

Veröffentlicht unter Software-Development | Verschlagwortet mit , | Hinterlasse einen Kommentar

Eclipse PDE Build: Unable to find plugin XYZ

Today I had the problem that suddenly my headless Eclipse PDE Build process wasn’t working anymore as soon as I added a new Bundle project to my feature.xml.

[java] /Applications/eclipse_3.5RC3/plugins/org.eclipse.pde.build_3.5.0.v20090527-1800/scripts/genericTargets.xml:106: Unable to find plug-in: bundlexyz

I first thought there is something wrong with the plugins MANIFEST.MF, but everything was ok.
It turned out that the new project I have created was in the wrong folder.

All my other bundle projects are inside a folder locally under /development/repo/myproject/plugins/bundle123 there is also my main ant build.xml.
and my Eclipse is under /development/workspace/ and
links to the repo folder.
Now, when I had created my new bundle project (which PDE Build complains about now) by accident I had forgotton to change the the location. I should have unchecked the checkbox „Use default location“ put the new plugin into /development/repo/myproject/plugins/bundlexyz instead of under my workspace folder.

1. Right click on your project and choose „Refactor > Move“
2. Choose the folder  /development/repo/myproject/plugins/bundle

After that change PDE Build stoppped complaining.

Remember: When using headless PDE Build, you have to put new plugins into the same location as all your other plugins. In Eclipse UI it will work, but in headless mode on the command line it will not work.

Veröffentlicht unter Software-Development | Verschlagwortet mit , , , | 1 Kommentar

„Ask the shortest question you can“

I found a really nice video by Dan Meyer who gives a presentation basically about what is wrong with the usual way we get taught math in school these days. And better than that he also gives advices on how to do it better.

Here are the 5 most important bullet points to me from his video:

  1. Use Multimedia
  2. Encourage Students intuition
  3. Ask the shortest question you can
  4. Let students build the problem
  5. Be less helpful.

To me this is the „less is more“ philosophy applied to teaching in general.

Veröffentlicht unter Allgemein | Verschlagwortet mit , | 41 Kommentare