Git for Windows with TortoiseGit and GitHub

For this project, we are required to use Git for version control and host code on GitHub. As a person who has always preferred SVN (for its linear/incremental nature), and have only had experience with Google Code and SourceForge, setting up a new GitHub repo was a new experience. And so, for future reference, I document the process here. In this article, we will install msysgit and TortoiseGit and host code on GitHub.

Installing TortoiseGit:

http://code.google.com/p/tortoisegit/ (yes, its hosted on Google Code) TortoiseGit is a shell extension for Windows Explorer that allows for easy management of Git projects. Its a port of the popular TortoiseSVN project, but for Git. First step would be installing Git for Windows (msysgit), which will also give you a nice, simplistic bash shell that you can work with (and execute the above commands just like on Linux if you'd prefer that over TortoiseGit). I personally just downloaded and installed Git 1.7.9 with all default settings. Next was installing TortoiseGit itself, which should autodetect the installed Git setup (though you can configure it later if you haven't) and add itself to the Windows Explorer right-click context menu.

Generating your public key:

To associate your computer with GitHub, you will need a "public key/private key" pair. You could do this entire process manually with the bash shell, but TortoiseGit gives you a handy GUI tool for it: Puttygen (PuTTY Key Generator). Run Puttygen via your Start menu: "Start -> Programs -> TortoiseGit -> Puttygen". Click "Generate", move your mouse around the blank box for a bit, and wait for the box to fill with your newly generated key. Copy the entire text in the box (right-click -> Select All, right-click -> Copy). This is your public key. Then click "Save private key" and choose a safe place to store the .pkk file (e.g. "TortoiseGit.ppk"), passphrase optional. This file stores your private key.

Adding your private key to TortoiseGit:

Next you want make your private key accessible to TortoiseGit for authenticating with GitHub. For this, we'll manage our key list with Pageant: "Start -> TortoiseGit -> Pageant". This will add an icon to your taskbar (see picture below). Right click the icon and select "View Keys". Click "Add Keys" and navigate to your previously saved PuTTY key that is storing your private key file (e.g. "D:\Development\Sandbox\TortoiseGit.ppk"). If added properly, you should see the new key added to the list.

Registering for GitHub:

https://github.com/signup/free
GitHub is a project hosting website that focuses mainly on Git. As of the moment, they offer free public repositories and unlimited public collaborators for open source projects. Registration is pretty straightforward - just sign up for a free personal account.

Adding your public key to GitHub:

https://github.com/settings/ssh
Go to your "Account Settings" on GitHub and go to the "SSH Keys" section. Click "Add SSH key" and paste your public key into the box. Give the key a descriptive title (it doesn't really matter) and then click "Add key". After confirming your GitHub password, you should see the new key added to your "SSH Keys" list.

Creating a GitHub repository:

https://github.com/repositories/new Go to your GitHub profile page (click your username on the top right) and click "New repository". Proceed to fill out the project information. Note that the "Project Name" becomes the repos name, so you may want to use a clean, space-less name (e.g. "PlanmanupNewGITProject" for this project). After clicking "Create repository", you'll be presented with a list of commands to execute to create your first commit. Since we'll be using TortoiseGit instead of a command-line to manage files, you can ignore

Creating your local repo

Now that the project has been created on GitHub, its time to create it on your computer! The GitHub page will give you bash commands for creating the new repos, but we'll use TortoiseGit instead. Navigate to the folder/drive where you want your code to be stored (e.g. "D:\Development\Sandbox"). In that folder, right click to open up the Windows Explorer context menu and select "Git clone...". In the popup box, copy-paste the Git repo URL from GitHub (see above picture) and click "OK". Since you've already set up the authentication earlier and its an empty repository, it should be quick (the log will warn you about cloning an empty repo though). If a new folder has been created with the name of your project (e.g. "D:\Development\Sandbox\PlanmanupNewGITProject"), the clone was successful!

Push files/changes to your local branch

What you have now is a clone of a branch of your repository. Your current branch is probably the master branch assuming you just created a clean project. Go ahead and add files and your code to the folder. Once you're happy with your code/changes, it's time to commit these changes. You can do so by right clicking to open up the context menu in the clone's folder and selecting "Git Commit -> "master"...". This will open up a window that will allow you to add the change message and select what changes you want to commit. If its your first commit, you will want to check the "Select/Deselect All" checkbox to add all your new files. Click "OK" when you're happy to commit the changes to your branch. Note that unlike SVN, this commit is LOCAL ONLY!

Synchronizing changes with GitHub

After a few commits, you might want to push your changes to GitHub. Open up the context menu in the clone's folder and selecting "TortoiseGit -> Push". In the new window, if you also added tags, don't forget to check the "Include Tags" checkbox before clicking "OK". If other project collaborators pushed out changes, you can pull their changes from GitHub with "TortoiseGit -> Pull".

Associating your private key with the git repos

To avoid having to manually start up the Pageant daemon every time you want to sync your git repos with GitHub, you can add your PuTTY key to TortoiseGit's Remote settings. Open up the context menu in the clone's folder and selecting "TortoiseGit -> Settings". In the new window, select "Git -> Remote" and select your Remote item (in my case, "origin"). Locate and select the .ppk file from earlier for the "Putty Key" field and click "OK". Now you don't have to manually start up Pageant every time you reboot your computer!

You're done! and that's it! If you check your GitHub project page, you should see your latest commits and navigate your code's different branches and tags.

Purge / Delete messages from JMS Weblogic

You can use JMX to purge the queue, either from Java or from WLST (Python). You can find the MBean definitions for WLS. Here is a basic Java program don't forget to put weblogic.jar in the CLASSPATH:

import java.util.Hashtable;
import javax.management.MBeanServerConnection;
import javax.management.remote.JMXConnector;
import javax.management.remote.JMXConnectorFactory;
import javax.management.remote.JMXServiceURL;
import javax.management.ObjectName;
import javax.naming.Context;
import weblogic.management.mbeanservers.runtime.RuntimeServiceMBean;

public class PurgeWLSQueue {
    private static final String WLS_USERNAME = "weblogic";
    private static final String WLS_PASSWORD = "weblogic";
    private static final String WLS_HOST = "localhost";
    private static final int WLS_PORT = 7001;
    private static final String JMS_SERVER = "wlsbJMSServer";
    private static final String JMS_DESTINATION = "test.q";
    private static JMXConnector getMBeanServerConnector(String jndiName) throws Exception {
        Hashtable<String,String> h = new Hashtable<String,String>();
        JMXServiceURL serviceURL = new JMXServiceURL("t3", WLS_HOST, WLS_PORT, jndiName);
        h.put(Context.SECURITY_PRINCIPAL, WLS_USERNAME);
        h.put(Context.SECURITY_CREDENTIALS, WLS_PASSWORD);
        h.put(JMXConnectorFactory.PROTOCOL_PROVIDER_PACKAGES, "weblogic.management.remote");
        JMXConnector connector = JMXConnectorFactory.connect(serviceURL, h);
        return connector;
    }
    public static void main(String[] args) {
        try {
            JMXConnector connector = getMBeanServerConnector("/jndi/"+RuntimeServiceMBean.MBEANSERVER_JNDI_NAME);
            MBeanServerConnection mbeanServerConnection = connector.getMBeanServerConnection();
            ObjectName service = new ObjectName("com.bea:Name=RuntimeService,Type=weblogic.management.mbeanservers.runtime.RuntimeServiceMBean");
            ObjectName serverRuntime = (ObjectName) mbeanServerConnection.getAttribute(service, "ServerRuntime");
            ObjectName jmsRuntime = (ObjectName) mbeanServerConnection.getAttribute(serverRuntime, "JMSRuntime");
            ObjectName[] jmsServers = (ObjectName[]) mbeanServerConnection.getAttribute(jmsRuntime, "JMSServers");
            for (ObjectName jmsServer: jmsServers) {
                if (JMS_SERVER.equals(jmsServer.getKeyProperty("Name"))) {
                    ObjectName[] destinations = (ObjectName[]) mbeanServerConnection.getAttribute(jmsServer, "Destinations");
                    for (ObjectName destination: destinations) {
                        if (destination.getKeyProperty("Name").endsWith("!"+JMS_DESTINATION)) {
                            Object o = mbeanServerConnection.invoke(destination,"deleteMessages",new Object[] {""},// selector expression
                                new String[] {"java.lang.String"});
                            System.out.println("Result: "+o);
                            break;
                        }
                    }
                    break;
                }
            }
            connector.close();
        } catch (Exception e) {
            e.printStackTrace();
        }
    }
}

Above solution works great on a single node environment, but what happens if you are on an clustered environment with ONE migratable JMSServer (currently on node #1) and this code is executing on node #2. Then there will be no JMSServer available and no message will be deleted.Solution:

ObjectName service = new ObjectName("com.bea:Name=DomainRuntimeService,Type=weblogic.management.mbeanservers.domainruntime.DomainRuntimeServiceMBean");

and be sure to access the admin port on the WLS-cluster.

Here is an example in WLST for a Managed Server running on port 7005:

connect('weblogic', 'weblogic', 't3://localhost:7005')
serverRuntime()
cd('/JMSRuntime/ManagedSrv1.jms/JMSServers/MyAppJMSServer/Destinations/MyAppJMSModule!QueueNameToClear')
cmo.deleteMessages('')

The last command should return the number of messages it deleted.

Removing the 128-bit key restriction in Java

Removing the 128-bit key restriction in Java
An issue in choosing an encryption key size in Java is that by default, current versions of the JDK have a deliberate key size restriction built in. If you try to perform, say, 256-bit AES encryption with the default JDK, you'll find that it dutifully throws an InvalidKeyException, complaining with the not-too-explicit message "Illegal key size or default parameters". If you get this exception, you're probably not doing anything wrong: You've just hit an arbitrary restriction imposed by (at least Sun's) JDK with default settings.

It turns out that the Cipher class will generally not allow encryption with a key size of more than 128 bits. The apparent reason behind this is that some countries (although increasingly fewer) have restrictions on the permitted key strength of imported encryption software, although the actual number 128 is questionable (see below). The good news is that:
You can easily remove the restriction by overriding the security policy files with others that Sun provides.Of course, by "easily", we mean "easy for somebody who doesn't mind downloading a zip, extracting some files from them and copying them to the right place inside the JRE folder". For some customers, this could make deployment a little impractical.

At present, the file you need is called Java Cryptography Extension (JCE) Unlimited Strength Jurisdiction Policy Files 6 and is currently available at the Java SE download page. This zip file contains a couple of policy jars, which you need copy over the top of the ones already in the lib/security directory of your JRE.

How to determine if a client has this restrictionYou can use Cipher.getMaxAllowedKeyLength() to return the maximum key length (in bits) permitted for a given algorithm. For example, to find out the maximum permitted key size with AES, we can call:
int maxKeyLen = Cipher.getMaxAllowedKeyLength("AES");
Note that this method generally returns the maximum key length permitted by policy, not necessarily by that algorithm!

Where did the number 128 come from?Now I have to confess, not being a politician, lawyer or lobotomised chimpanzee, I don't fully understand the rationale behind the number 128, for at least two reasons:

• It's difficult to actually find a reference to any country's current import law that specifically mentions the magic number 128 (see Bert-Jaap Koops' cryptography law survey). 
• In any case, the JDK ships with a 160-bit secure hash algorithm and a random number generator built on this algorithm. (Remember, you can create a stream cipher by XORing your data with a secure random number source; however strong or weak you argue it is, SecureRandom can in effect serve as a 160-bit encryption algorithm.)

Tuning the compaction and thread local area (TLA) size.

1. Compaction is used to reduce heap fragmentation, i.e., move objects to form contiguous ‘live regions’ of the heap space. Note when a full compaction is run the stopping of a concurrently running application is inevitable. We can say that compaction is one of the worst garbage collection bottlenecks. If anything is known about fragmentation and object sizes it might be beneficial to tune compaction.
2. The compaction algorithm divides the heap into a number of equally large parts. Each of these is subject to separate compaction that can stop the world. The default is 4096 parts. When compaction is too disruptive it might help to increase the number of heap parts. When compaction fails to keep up with fragmentation it might help to decrease the number of heap parts. Note that, for strategies other than throughput the compaction areas are sized dynamically.
3. The proper tuning of compaction involves two parameters:
1. Compaction ratio (-XX:compaction:percentage:<percentage> or -XX:compaction:internalPercentage=<percentage>,externalPercentage=<percentage>) – the percentage of the heap that the garbage collector compacts at each garbage collection. While the JVM is compacting the heap, all threads that want to access objects need to wait because the JVM is moving the objects around.
2. Maximum references (-XX:compaction:maxReferences:<value>) – the maximum number of references to objects in the compaction area. If the number of references exceeds this limit, the compaction is canceled. When compaction has moved objects, the references to these objects must be updated. The pause time introduced by this updating is proprotional to the number of references that have been updated.
4. Steps to tune compaction are as follows:
1. Set the compaction ratio to 1 and gradually increase the ratio until the pause time becomes too long.
2. If the garbage collection time is too long when the compaction ration is 1, the maximum number of references must be adjusted.
3. Set the maximum references to 10000 and gradually increase the references until the pause time becomes too long. To monitor compaction behavior, we can add the option -Xverbose:compaction to the command-line. If many compactions are skipped, the maximum references must be increased; if, on the other hand, the compaction pause times are too long the maximum references must be decreased.
4. As we are using a dynamic garbage collection that optimizes the pause time (-Xgc:pausetime), we do not need to tune the compaction manually in this case setting the -XpauseTarget is sufficient.
5. Each thread allocates objects in a TLA that is promoted to the heap when full. The TLA size can be adjusted by using -XXtlaSize:min=<size>,preferred=<size>,wasteLimit=<size>, in which:
1. min – sets the minimum size of a TLA.
2. preferred – sets the preferred size of a TLA. The system will try to get TLAs of the preferred size if possible, but accepts TLAs of minimum size as well.
3. wasteLimit – sets the waste limit for TLAs. This is the maximum amount of free memory that a TLA is allowed to have when a thread requires a new TLA

Java Weblogic SSL

This is a simple WebLogic SSL configuration. If you’re not using WebLogic with SSL you probably should be. Minimally have SSL setup to encrypt your passwords to the administration consoles. I actually force SSL and disable the standard http listen port. My authentication provider is active directory and I’m using SSL there as well. The node manager is also setup using SSL as are the managed servers (JVMs). I’ll get into all that a bit later.

I’ll be using java’s keytool for all of this. We will create two keystores, one for the identity and one for the trust. You could also use the standard java trust and simply add your root certificates to it.
Some Notes (WebLogic Default Keystore Passwords - In case you want to mess with the demo trust or demo keystore)
Trust store password: DemoTrustKeyStorePassPhrase
Key store password: DemoIdentityKeyStorePassPhrase
Private key password: DemoIdentityPassPhrase
Java standard trust store password: changeit

Using java keytool:
Create the identity keystore and keypair.
I cd directly to the directory where WebLogic stores its demo trust and demo identity keystores. In my case /opt/oracle/middleware/wlserver_10.3/server/lib. My two keystores weblogic_identity.jks and weblogic_identity.jks will be created and stored there.

/opt/oracle/middleware/java/bin/keytool -genkey -alias weblogicServer -keyalg RSA -keysize 2048 -keystore weblogic_identity.jks -dname "CN=myhost.domain.com,OU=Middleware, O=MyOrg"

You will be asked to create a password for this keystore, so make sure to save it or remember it.

Create the certificate signing request (CSR):

/opt/oracle/middleware/java/bin/keytool -certreq -alias weblogicServer -file myhost.csr -keystore weblogic_identity.jks

Take the contents of the myhost.csr and submit it to your internal certificate authority (CA) or another external CA. In my example I get three certificates back. The root certificate, the intermediate certificate and the newly signed certificate we just submitted our CSR for. I get all of these back in base 64 encoding. Once you have these you can begin importing them into the proper keystores.

Create the trust keystore & import the root certificate:

/opt/oracle/middleware/java/bin/keytool -import -trustcacerts -alias myRoot -file /path/to/myRoot.cer  -keystore weblogic_trust.jks

Import intermediate certificate to trust keystore

/opt/oracle/middleware/java/bin/keytool -import -trustcacerts -alias entRoot -file /path/to/entRoot.cer  -keystore weblogic_trust.jks

Import root certificate to identity keystore

/opt/oracle/middleware/java/bin/keytool -import -trustcacerts -alias myRoot -file /path/to/myRoot.cer  -keystore weblogic_identity.jks

Import intermediate certificate to identity keystore:

/opt/oracle/middleware/java/bin/keytool -import -trustcacerts -alias entRoot -file /path/to/entRoot.cer  -keystore weblogic_identity.jks

Import signed certicifate to identity keystore:

/opt/oracle/middleware/java/bin/keytool -import -trustcacerts -alias weblogicServer -file /path/to/mySignedCert.cer -keystore weblogic_identity.jks

That’s all we have to do with keytool. We now have the two java keystores we need to configure Weblogic SSL.
WebLogic WSLT script for SSLThis script will setup the above keystores for your admin and all your managed JVMs.

#!/usr/bin/python
# Read Properties File
loadProperties("/path/to/scripts/my.props")

# Split if more than one.
WLmgdName = WLmgdNameList.split(',')
 
# Connect String
connect(username,password,'t3://'+adminHost+':'+adminPort)
 
# Get your edit on son! DO WORK!
edit()
startEdit()
 
# Admin Server SSL & Keystore
cd('/Servers/'+adminName)
cmo.setKeyStores('CustomIdentityAndCustomTrust')
cmo.setCustomIdentityKeyStoreFileName(wlHome+'/server/lib/weblogic_identity.jks')
cmo.setCustomIdentityKeyStoreType('jks')
cmo.setCustomTrustKeyStoreFileName(wlHome+'/server/lib/weblogic_trust.jks')
cmo.setCustomTrustKeyStoreType('jks')
cd('/Servers/'+adminName+'/SSL/'+adminName)
cmo.setServerPrivateKeyAlias('weblogicServer')
 
for mgdServer in WLmgdName:
       # Managed Server SSL & Keystore
       cd('/Servers/'+mgdServer)
       cmo.setKeyStores('CustomIdentityAndCustomTrust')
       cmo.setCustomIdentityKeyStoreFileName(wlHome+'/server/lib/weblogic_identity.jks')
       cmo.setCustomIdentityKeyStoreType('jks')
       cmo.setCustomTrustKeyStoreFileName(wlHome+'/server/lib/weblogic_trust.jks')
       cmo.setCustomTrustKeyStoreType('jks')
       cd('/Servers/'+mgdServer+'/SSL/'+mgdServer)
       cmo.setServerPrivateKeyAlias('weblogicServer')
 
save()
activate()
exit()

WLST properties file (my.props)

username=weblogic
password=weblogic123
adminName=my_admin
adminHost=myadmin.domain.com
adminPort=30000
WLmgdNameList=jvm01,jvm02,jvm03,jvm04,jvm05,jvm06
wlHome=/opt/oracle/middleware/wlserver_10.3

Now you should log into the admin console and change your passwords under Servers SSL and Keystores to use the password I told you to save or remember back at the start of this post. You can probably add the password bits to the script if you want, I’ll have to check that out.Weblogic node manager SSL
Edit nodemanager.properties
You should make sure you are using SecureListener=true and add the following:

KeyStores=CustomIdentityAndCustomTrust
CustomIdentityKeyStoreFileName=/opt/oracle/middleware/wlserver_10.3/server/lib/weblogic_identity.jks
CustomIdentityKeyStorePassPhrase=t0ps3cret
CustomIdentityAlias=weblogicServer
CustomIdentityPrivateKeyPassPhrase=t0ps3cret
CustomTrustKeyStoreFileName=/opt/oracle/middleware/wlserver_10.3/server/lib/weblogic_trust.jks

These passwords will encrypt on first start.
Set node manager type to SSL
Log into your admin console and make sure node manager type is set to SSL.
After all that make sure you save and activate any changes you made and restart everything and you should be good to go. Rock on…..

Weblogic Cluster SSL:
Change cluster address port
I assign addresses and ports on my cluster page, don’t forget to change the port to your secure port.
Secure Replication
Oh yea, if you disable all your regular listen ports and change cluster communication to use SSL make sure you change your cluster replication to “Secure Replication Enabled” or else things wont work. This setting is under clusters > cluster name > replication. You will see an error similar to:

server subsystem failed. Reason: java.lang.AssertionError: No replication server channel for osb_01 java.lang.AssertionError: No replication server channel for osb_01

HOW TO CLEAR WLI_PROCESS_EVENT

Can I clear WLI_PROCESS_EVENT, the system is using 29 GB of LOB space and I have an idea that WLI_PROCESS_EVENT is responsible for it. The reason of clearing is DB is getting full and we don't want historic data, and when server crash it took longer to come back.

Some Counts from the DB (Those with CLOB's/BLOB's):
SELECT COUNT(*) FROM WLI_CALENDAR
-- 1
SELECT COUNT(*) FROM WLI_PROCESS_DOCUMENT
-- 12
SELECT COUNT(*) FROM WLI_PROCESS_EVENT
-- 638564
SELECT COUNT(*) FROM WLI_PROCESS_TRACKING
-- 81
SELECT COUNT(*) FROM WLI_WORKLIST_DATA
-- 0
SELECT COUNT(*) FROM WLI_MT_CONTENT 
-- 0
SELECT COUNT(*) FROM WLI_PROCESS_INSTANCE_INFO;
-- 112

Solution: There are large amount of orphaned events which has a LOB column, so basically get rid of them:
Run this SQL Query and commit

$ Update  <DB_SCHEMA_NAME>.WLI_PROCESS_INSTANCE_INFO SET PROCESS_STATUS = 5 WHERE PROCESS_STATUS = 1;

Go to wliconsole and do a manual purge. Run this SQL Query and commit:

Run this SQL Query and commit:
DELETE FROM WLI_PROCESS_EVENT WHERE PROCESS_INSTANCE IN (SELECT WLI_PROCESS_EVENT.PROCESS_INSTANCE FROM WLI_PROCESS_EVENT LEFT OUTER JOIN WLI_PROCESS_INSTANCE_INFO ON WLI_PROCESS_EVENT.PROCESS_INSTANCE = WLI_PROCESS_INSTANCE_INFO.PROCESS_INSTANCE WHERE WLI_PROCESS_INSTANCE_INFO.PROCESS_INSTANCE is NULL)

NOTE: Because I'm using Oracle, the LOB segment did not shrink, it only freed up blocks. To free up space I need to drop the column (which will drop the lobsgment) and recreate the column OR truncate the table.

How to enable GUI while connecting to Remote Redhat / Linux machine using Putty.

This post cover details about how we can enable GUI interfacing using Xserver while connecting to Remote Redhat 5.6 / Linux Machine using Putty on Windows based local machine.

First all of you need to install Xserver in your local box. Xserver will be installed using Xming.

After downloading install the Xming server in your localbox and run it with option ":0 -clipboard -multiwindow -ac". To do this, right click the short cut of Xming -> go to properties -> and in target  its should look similar to this "C:\Program Files\Xming\Xming.exe" :0 -clipboard -multiwindow -ac (here double quates are part of the string itself), depending on the location of Xming installation path may change.

On Redhat / Linux machine has feature called X11Forwarding, depending on the value of this parameter it enables or disables the display of graphics on the server.

1. login on Linux/BSD system called myserver.mydomain.com
2. Open /etc/ssh/sshd_config file using text editor:
1. # vi /etc/ssh/sshd_config
3. Find out parameter X11Forwarding and set it to yes:
1. X11Forwarding yes
2. Save file & exit shell prompt.
4. Restart sshd service under Debian Linux:
1. # /etc/init.d/ssh restart
2. Alternatively, if you are using Fedora / Red Hat Linux restart sshd:
3. # /etc/init.d/sshd restart

Client Side Setting on Windows Machine
For connecting to Redhat Linux box I am using the most popular SSH client putty. Below are the steps to configure putty on Windows machine.

1. Run the putty.exe
2. provide Host Name (or you can use IP address of host machine as well)
3. Select SSH as Connection Type.
4. Port should be 22 default
5. Enter again the same name as you entered in Hostname in to Saved Session Input box.
6. In Connection Category, Find out the Connection Tree. In SSH, expand it and you will see "Enable X11 Forwarding"..
7. Enable X11 Forwarding by selecting the check box
8. X Display location  should be set to localhost:0
9. Save this entire information as a session by click on Save button
10. Now start the Xming Server on location machine
11. Now connect to the Redhat / Linux Machine using saved session from putty
12. And to verify that Graphics are enable use this command xclock &
13. You should be able to new graphical window coming up.

Now you are all set to roll, you can execute and run any GUI based application from the Redhat / Linux box and it will get displayed on you local machine.