Purge / Delete messages from JMS Weblogic

You can use JMX to purge the queue, either from Java or from WLST (Python). You can find the MBean definitions for WLS. Here is a basic Java program don't forget to put weblogic.jar in the CLASSPATH:

import java.util.Hashtable;
import javax.management.MBeanServerConnection;
import javax.management.remote.JMXConnector;
import javax.management.remote.JMXConnectorFactory;
import javax.management.remote.JMXServiceURL;
import javax.management.ObjectName;
import javax.naming.Context;
import weblogic.management.mbeanservers.runtime.RuntimeServiceMBean;

public class PurgeWLSQueue {
    private static final String WLS_USERNAME = "weblogic";
    private static final String WLS_PASSWORD = "weblogic";
    private static final String WLS_HOST = "localhost";
    private static final int WLS_PORT = 7001;
    private static final String JMS_SERVER = "wlsbJMSServer";
    private static final String JMS_DESTINATION = "test.q";
    private static JMXConnector getMBeanServerConnector(String jndiName) throws Exception {
        Hashtable<String,String> h = new Hashtable<String,String>();
        JMXServiceURL serviceURL = new JMXServiceURL("t3", WLS_HOST, WLS_PORT, jndiName);
        h.put(Context.SECURITY_PRINCIPAL, WLS_USERNAME);
        h.put(Context.SECURITY_CREDENTIALS, WLS_PASSWORD);
        h.put(JMXConnectorFactory.PROTOCOL_PROVIDER_PACKAGES, "weblogic.management.remote");
        JMXConnector connector = JMXConnectorFactory.connect(serviceURL, h);
        return connector;
    }
    public static void main(String[] args) {
        try {
            JMXConnector connector = getMBeanServerConnector("/jndi/"+RuntimeServiceMBean.MBEANSERVER_JNDI_NAME);
            MBeanServerConnection mbeanServerConnection = connector.getMBeanServerConnection();
            ObjectName service = new ObjectName("com.bea:Name=RuntimeService,Type=weblogic.management.mbeanservers.runtime.RuntimeServiceMBean");
            ObjectName serverRuntime = (ObjectName) mbeanServerConnection.getAttribute(service, "ServerRuntime");
            ObjectName jmsRuntime = (ObjectName) mbeanServerConnection.getAttribute(serverRuntime, "JMSRuntime");
            ObjectName[] jmsServers = (ObjectName[]) mbeanServerConnection.getAttribute(jmsRuntime, "JMSServers");
            for (ObjectName jmsServer: jmsServers) {
                if (JMS_SERVER.equals(jmsServer.getKeyProperty("Name"))) {
                    ObjectName[] destinations = (ObjectName[]) mbeanServerConnection.getAttribute(jmsServer, "Destinations");
                    for (ObjectName destination: destinations) {
                        if (destination.getKeyProperty("Name").endsWith("!"+JMS_DESTINATION)) {
                            Object o = mbeanServerConnection.invoke(destination,"deleteMessages",new Object[] {""},// selector expression
                                new String[] {"java.lang.String"});
                            System.out.println("Result: "+o);
                            break;
                        }
                    }
                    break;
                }
            }
            connector.close();
        } catch (Exception e) {
            e.printStackTrace();
        }
    }
}

Above solution works great on a single node environment, but what happens if you are on an clustered environment with ONE migratable JMSServer (currently on node #1) and this code is executing on node #2. Then there will be no JMSServer available and no message will be deleted.Solution:

ObjectName service = new ObjectName("com.bea:Name=DomainRuntimeService,Type=weblogic.management.mbeanservers.domainruntime.DomainRuntimeServiceMBean");

and be sure to access the admin port on the WLS-cluster.

Here is an example in WLST for a Managed Server running on port 7005:

connect('weblogic', 'weblogic', 't3://localhost:7005')
serverRuntime()
cd('/JMSRuntime/ManagedSrv1.jms/JMSServers/MyAppJMSServer/Destinations/MyAppJMSModule!QueueNameToClear')
cmo.deleteMessages('')

The last command should return the number of messages it deleted.

Removing the 128-bit key restriction in Java

Removing the 128-bit key restriction in Java
An issue in choosing an encryption key size in Java is that by default, current versions of the JDK have a deliberate key size restriction built in. If you try to perform, say, 256-bit AES encryption with the default JDK, you'll find that it dutifully throws an InvalidKeyException, complaining with the not-too-explicit message "Illegal key size or default parameters". If you get this exception, you're probably not doing anything wrong: You've just hit an arbitrary restriction imposed by (at least Sun's) JDK with default settings.

It turns out that the Cipher class will generally not allow encryption with a key size of more than 128 bits. The apparent reason behind this is that some countries (although increasingly fewer) have restrictions on the permitted key strength of imported encryption software, although the actual number 128 is questionable (see below). The good news is that:
You can easily remove the restriction by overriding the security policy files with others that Sun provides.Of course, by "easily", we mean "easy for somebody who doesn't mind downloading a zip, extracting some files from them and copying them to the right place inside the JRE folder". For some customers, this could make deployment a little impractical.

At present, the file you need is called Java Cryptography Extension (JCE) Unlimited Strength Jurisdiction Policy Files 6 and is currently available at the Java SE download page. This zip file contains a couple of policy jars, which you need copy over the top of the ones already in the lib/security directory of your JRE.

How to determine if a client has this restrictionYou can use Cipher.getMaxAllowedKeyLength() to return the maximum key length (in bits) permitted for a given algorithm. For example, to find out the maximum permitted key size with AES, we can call:
int maxKeyLen = Cipher.getMaxAllowedKeyLength("AES");
Note that this method generally returns the maximum key length permitted by policy, not necessarily by that algorithm!

Where did the number 128 come from?Now I have to confess, not being a politician, lawyer or lobotomised chimpanzee, I don't fully understand the rationale behind the number 128, for at least two reasons:

• It's difficult to actually find a reference to any country's current import law that specifically mentions the magic number 128 (see Bert-Jaap Koops' cryptography law survey). 
• In any case, the JDK ships with a 160-bit secure hash algorithm and a random number generator built on this algorithm. (Remember, you can create a stream cipher by XORing your data with a secure random number source; however strong or weak you argue it is, SecureRandom can in effect serve as a 160-bit encryption algorithm.)

Tuning the compaction and thread local area (TLA) size.

1. Compaction is used to reduce heap fragmentation, i.e., move objects to form contiguous ‘live regions’ of the heap space. Note when a full compaction is run the stopping of a concurrently running application is inevitable. We can say that compaction is one of the worst garbage collection bottlenecks. If anything is known about fragmentation and object sizes it might be beneficial to tune compaction.
2. The compaction algorithm divides the heap into a number of equally large parts. Each of these is subject to separate compaction that can stop the world. The default is 4096 parts. When compaction is too disruptive it might help to increase the number of heap parts. When compaction fails to keep up with fragmentation it might help to decrease the number of heap parts. Note that, for strategies other than throughput the compaction areas are sized dynamically.
3. The proper tuning of compaction involves two parameters:
1. Compaction ratio (-XX:compaction:percentage:<percentage> or -XX:compaction:internalPercentage=<percentage>,externalPercentage=<percentage>) – the percentage of the heap that the garbage collector compacts at each garbage collection. While the JVM is compacting the heap, all threads that want to access objects need to wait because the JVM is moving the objects around.
2. Maximum references (-XX:compaction:maxReferences:<value>) – the maximum number of references to objects in the compaction area. If the number of references exceeds this limit, the compaction is canceled. When compaction has moved objects, the references to these objects must be updated. The pause time introduced by this updating is proprotional to the number of references that have been updated.
4. Steps to tune compaction are as follows:
1. Set the compaction ratio to 1 and gradually increase the ratio until the pause time becomes too long.
2. If the garbage collection time is too long when the compaction ration is 1, the maximum number of references must be adjusted.
3. Set the maximum references to 10000 and gradually increase the references until the pause time becomes too long. To monitor compaction behavior, we can add the option -Xverbose:compaction to the command-line. If many compactions are skipped, the maximum references must be increased; if, on the other hand, the compaction pause times are too long the maximum references must be decreased.
4. As we are using a dynamic garbage collection that optimizes the pause time (-Xgc:pausetime), we do not need to tune the compaction manually in this case setting the -XpauseTarget is sufficient.
5. Each thread allocates objects in a TLA that is promoted to the heap when full. The TLA size can be adjusted by using -XXtlaSize:min=<size>,preferred=<size>,wasteLimit=<size>, in which:
1. min – sets the minimum size of a TLA.
2. preferred – sets the preferred size of a TLA. The system will try to get TLAs of the preferred size if possible, but accepts TLAs of minimum size as well.
3. wasteLimit – sets the waste limit for TLAs. This is the maximum amount of free memory that a TLA is allowed to have when a thread requires a new TLA

Java Weblogic SSL

This is a simple WebLogic SSL configuration. If you’re not using WebLogic with SSL you probably should be. Minimally have SSL setup to encrypt your passwords to the administration consoles. I actually force SSL and disable the standard http listen port. My authentication provider is active directory and I’m using SSL there as well. The node manager is also setup using SSL as are the managed servers (JVMs). I’ll get into all that a bit later.

I’ll be using java’s keytool for all of this. We will create two keystores, one for the identity and one for the trust. You could also use the standard java trust and simply add your root certificates to it.
Some Notes (WebLogic Default Keystore Passwords - In case you want to mess with the demo trust or demo keystore)
Trust store password: DemoTrustKeyStorePassPhrase
Key store password: DemoIdentityKeyStorePassPhrase
Private key password: DemoIdentityPassPhrase
Java standard trust store password: changeit

Using java keytool:
Create the identity keystore and keypair.
I cd directly to the directory where WebLogic stores its demo trust and demo identity keystores. In my case /opt/oracle/middleware/wlserver_10.3/server/lib. My two keystores weblogic_identity.jks and weblogic_identity.jks will be created and stored there.

/opt/oracle/middleware/java/bin/keytool -genkey -alias weblogicServer -keyalg RSA -keysize 2048 -keystore weblogic_identity.jks -dname "CN=myhost.domain.com,OU=Middleware, O=MyOrg"

You will be asked to create a password for this keystore, so make sure to save it or remember it.

Create the certificate signing request (CSR):

/opt/oracle/middleware/java/bin/keytool -certreq -alias weblogicServer -file myhost.csr -keystore weblogic_identity.jks

Take the contents of the myhost.csr and submit it to your internal certificate authority (CA) or another external CA. In my example I get three certificates back. The root certificate, the intermediate certificate and the newly signed certificate we just submitted our CSR for. I get all of these back in base 64 encoding. Once you have these you can begin importing them into the proper keystores.

Create the trust keystore & import the root certificate:

/opt/oracle/middleware/java/bin/keytool -import -trustcacerts -alias myRoot -file /path/to/myRoot.cer  -keystore weblogic_trust.jks

Import intermediate certificate to trust keystore

/opt/oracle/middleware/java/bin/keytool -import -trustcacerts -alias entRoot -file /path/to/entRoot.cer  -keystore weblogic_trust.jks

Import root certificate to identity keystore

/opt/oracle/middleware/java/bin/keytool -import -trustcacerts -alias myRoot -file /path/to/myRoot.cer  -keystore weblogic_identity.jks

Import intermediate certificate to identity keystore:

/opt/oracle/middleware/java/bin/keytool -import -trustcacerts -alias entRoot -file /path/to/entRoot.cer  -keystore weblogic_identity.jks

Import signed certicifate to identity keystore:

/opt/oracle/middleware/java/bin/keytool -import -trustcacerts -alias weblogicServer -file /path/to/mySignedCert.cer -keystore weblogic_identity.jks

That’s all we have to do with keytool. We now have the two java keystores we need to configure Weblogic SSL.
WebLogic WSLT script for SSLThis script will setup the above keystores for your admin and all your managed JVMs.

#!/usr/bin/python
# Read Properties File
loadProperties("/path/to/scripts/my.props")

# Split if more than one.
WLmgdName = WLmgdNameList.split(',')
 
# Connect String
connect(username,password,'t3://'+adminHost+':'+adminPort)
 
# Get your edit on son! DO WORK!
edit()
startEdit()
 
# Admin Server SSL & Keystore
cd('/Servers/'+adminName)
cmo.setKeyStores('CustomIdentityAndCustomTrust')
cmo.setCustomIdentityKeyStoreFileName(wlHome+'/server/lib/weblogic_identity.jks')
cmo.setCustomIdentityKeyStoreType('jks')
cmo.setCustomTrustKeyStoreFileName(wlHome+'/server/lib/weblogic_trust.jks')
cmo.setCustomTrustKeyStoreType('jks')
cd('/Servers/'+adminName+'/SSL/'+adminName)
cmo.setServerPrivateKeyAlias('weblogicServer')
 
for mgdServer in WLmgdName:
       # Managed Server SSL & Keystore
       cd('/Servers/'+mgdServer)
       cmo.setKeyStores('CustomIdentityAndCustomTrust')
       cmo.setCustomIdentityKeyStoreFileName(wlHome+'/server/lib/weblogic_identity.jks')
       cmo.setCustomIdentityKeyStoreType('jks')
       cmo.setCustomTrustKeyStoreFileName(wlHome+'/server/lib/weblogic_trust.jks')
       cmo.setCustomTrustKeyStoreType('jks')
       cd('/Servers/'+mgdServer+'/SSL/'+mgdServer)
       cmo.setServerPrivateKeyAlias('weblogicServer')
 
save()
activate()
exit()

WLST properties file (my.props)

username=weblogic
password=weblogic123
adminName=my_admin
adminHost=myadmin.domain.com
adminPort=30000
WLmgdNameList=jvm01,jvm02,jvm03,jvm04,jvm05,jvm06
wlHome=/opt/oracle/middleware/wlserver_10.3

Now you should log into the admin console and change your passwords under Servers SSL and Keystores to use the password I told you to save or remember back at the start of this post. You can probably add the password bits to the script if you want, I’ll have to check that out.Weblogic node manager SSL
Edit nodemanager.properties
You should make sure you are using SecureListener=true and add the following:

KeyStores=CustomIdentityAndCustomTrust
CustomIdentityKeyStoreFileName=/opt/oracle/middleware/wlserver_10.3/server/lib/weblogic_identity.jks
CustomIdentityKeyStorePassPhrase=t0ps3cret
CustomIdentityAlias=weblogicServer
CustomIdentityPrivateKeyPassPhrase=t0ps3cret
CustomTrustKeyStoreFileName=/opt/oracle/middleware/wlserver_10.3/server/lib/weblogic_trust.jks

These passwords will encrypt on first start.
Set node manager type to SSL
Log into your admin console and make sure node manager type is set to SSL.
After all that make sure you save and activate any changes you made and restart everything and you should be good to go. Rock on…..

Weblogic Cluster SSL:
Change cluster address port
I assign addresses and ports on my cluster page, don’t forget to change the port to your secure port.
Secure Replication
Oh yea, if you disable all your regular listen ports and change cluster communication to use SSL make sure you change your cluster replication to “Secure Replication Enabled” or else things wont work. This setting is under clusters > cluster name > replication. You will see an error similar to:

server subsystem failed. Reason: java.lang.AssertionError: No replication server channel for osb_01 java.lang.AssertionError: No replication server channel for osb_01

HOW TO CLEAR WLI_PROCESS_EVENT

Can I clear WLI_PROCESS_EVENT, the system is using 29 GB of LOB space and I have an idea that WLI_PROCESS_EVENT is responsible for it. The reason of clearing is DB is getting full and we don't want historic data, and when server crash it took longer to come back.

Some Counts from the DB (Those with CLOB's/BLOB's):
SELECT COUNT(*) FROM WLI_CALENDAR
-- 1
SELECT COUNT(*) FROM WLI_PROCESS_DOCUMENT
-- 12
SELECT COUNT(*) FROM WLI_PROCESS_EVENT
-- 638564
SELECT COUNT(*) FROM WLI_PROCESS_TRACKING
-- 81
SELECT COUNT(*) FROM WLI_WORKLIST_DATA
-- 0
SELECT COUNT(*) FROM WLI_MT_CONTENT 
-- 0
SELECT COUNT(*) FROM WLI_PROCESS_INSTANCE_INFO;
-- 112

Solution: There are large amount of orphaned events which has a LOB column, so basically get rid of them:
Run this SQL Query and commit

$ Update  <DB_SCHEMA_NAME>.WLI_PROCESS_INSTANCE_INFO SET PROCESS_STATUS = 5 WHERE PROCESS_STATUS = 1;

Go to wliconsole and do a manual purge. Run this SQL Query and commit:

Run this SQL Query and commit:
DELETE FROM WLI_PROCESS_EVENT WHERE PROCESS_INSTANCE IN (SELECT WLI_PROCESS_EVENT.PROCESS_INSTANCE FROM WLI_PROCESS_EVENT LEFT OUTER JOIN WLI_PROCESS_INSTANCE_INFO ON WLI_PROCESS_EVENT.PROCESS_INSTANCE = WLI_PROCESS_INSTANCE_INFO.PROCESS_INSTANCE WHERE WLI_PROCESS_INSTANCE_INFO.PROCESS_INSTANCE is NULL)

NOTE: Because I'm using Oracle, the LOB segment did not shrink, it only freed up blocks. To free up space I need to drop the column (which will drop the lobsgment) and recreate the column OR truncate the table.

How to enable GUI while connecting to Remote Redhat / Linux machine using Putty.

This post cover details about how we can enable GUI interfacing using Xserver while connecting to Remote Redhat 5.6 / Linux Machine using Putty on Windows based local machine.

First all of you need to install Xserver in your local box. Xserver will be installed using Xming.

After downloading install the Xming server in your localbox and run it with option ":0 -clipboard -multiwindow -ac". To do this, right click the short cut of Xming -> go to properties -> and in target  its should look similar to this "C:\Program Files\Xming\Xming.exe" :0 -clipboard -multiwindow -ac (here double quates are part of the string itself), depending on the location of Xming installation path may change.

On Redhat / Linux machine has feature called X11Forwarding, depending on the value of this parameter it enables or disables the display of graphics on the server.

1. login on Linux/BSD system called myserver.mydomain.com
2. Open /etc/ssh/sshd_config file using text editor:
1. # vi /etc/ssh/sshd_config
3. Find out parameter X11Forwarding and set it to yes:
1. X11Forwarding yes
2. Save file & exit shell prompt.
4. Restart sshd service under Debian Linux:
1. # /etc/init.d/ssh restart
2. Alternatively, if you are using Fedora / Red Hat Linux restart sshd:
3. # /etc/init.d/sshd restart

Client Side Setting on Windows Machine
For connecting to Redhat Linux box I am using the most popular SSH client putty. Below are the steps to configure putty on Windows machine.

1. Run the putty.exe
2. provide Host Name (or you can use IP address of host machine as well)
3. Select SSH as Connection Type.
4. Port should be 22 default
5. Enter again the same name as you entered in Hostname in to Saved Session Input box.
6. In Connection Category, Find out the Connection Tree. In SSH, expand it and you will see "Enable X11 Forwarding"..
7. Enable X11 Forwarding by selecting the check box
8. X Display location  should be set to localhost:0
9. Save this entire information as a session by click on Save button
10. Now start the Xming Server on location machine
11. Now connect to the Redhat / Linux Machine using saved session from putty
12. And to verify that Graphics are enable use this command xclock &
13. You should be able to new graphical window coming up.

Now you are all set to roll, you can execute and run any GUI based application from the Redhat / Linux box and it will get displayed on you local machine.

How To Encrypt Clear Text Passwords With WebLogic Server

WebLogic Server encrypts all the plain text passwords stored in its domain configuration XML file(s). This is to prevent access to sensitive information. When passwords are entered using administration console or scripting tools, it will automatically get encrypted before they are stored in the configuration XML files(s).

Prior to WebLogic Server 9.0: If those passwords need to be reset either the configuration tools (Console or scripting tools) can be used which will automatically re-encrypt the passwords or by directly changing the configuration files using a text editor. When files are directly modified using a text editor the passwords will get encrypted during the subsequent restart.

Starting from WebLogic Server 9.0: Using clear text passwords in the configuration files are supported only for Development domain and it will not re-encrypt the passwords. If the domain is a Production domain then you cannot set the passwords in clear text. You have to either use a dedicated command-line utility or WLST to encrypt the clear text passwords. If the server encounters a clear text password when parsing the configuration file(s) while starting in Production Mode, then you will get an error similar to the following:
<Oct 20, 2008 9:05:35 PM UTC> <Critical> <WebLogicServer> <BEA-000362> <Server failed. Reason: [Management:141266]Parsing Failure in config.xml: java.lang.IllegalArgumentException: In production mode, it's not allowed to set a clear text value to the property: PasswordEncrypted of ServerStartMBean>
Depending on the configuration the MBean name value of the error message may change. In this case the ServerStartMBean has clear text value for a password property. Either the dedicated Java utility to encrypt clear text values can be used or WLST cant be used to re-encrypt. To run the encrypt utility follow the instructions below:

1. Change directory to your domain's bin folder (For Eg. cd c:\bea\user_projects\domains\mydomain\bin)
2. Execute the setDomainEnv script (For Eg. setDomainEnv.cmd)
3. Execute java weblogic.security.Encrypt which will prompt for the password and will print the encrypted value in stdout.
4. The following are some sample output from running the utility
1. C:\bea\user_projects\domains\mydomain>java weblogic.security.Encrypt
   Password:
   {3DES}9HWsf87pJTw=
2. You should execute this utility from the domain folder as it requires the domain's password salt file (SerializedSystemIni.dat) for encrypting the clear text string. You can also pass the clear text string as an argument: C:\bea\user_projects\domains\mydomain>java weblogic.security.Encrypt testpwd
   {3DES}9HWsf87pJTw=
5. You can also use WLST to encrypt clear text strings as below:C:\bea\user_projects\domains\mydomain>java weblogic.WLST
   Initializing WebLogic Scripting Tool (WLST) ...
   Welcome to WebLogic Server Administration Scripting Shell
   Type help() for help on available commands
   wls:/offline> es = encrypt('testpwd')
   wls:/offline> print es
   {3DES}9HWsf87pJTw=
   wls:/offline>
1. When running WLST from a location different than the domain folder you can pass in an argument to specify the domain directory. Once you have the encrypted value, the configuration files can be modified to include this encrypte value instead of clear text passwords. These features will make your domain to operate when resetting the encrypted passwords on a Production domain's configuration XML files. These methods not only can be used to encrypt configuration XML (config.xml) but also the JDBC or JMS descriptor XML files.