Search This Blog

Wednesday, February 8, 2017

The Garbage Collector [JAVA]

Java Memory Management, with its built-in garbage collection, is one of the language’s finest achievements. It allows developers to create new objects without worrying explicitly about memory allocation and deallocation, because the garbage collector automatically reclaims memory for reuse. This enables faster development with less boilerplate code, while eliminating memory leaks and other memory-related problems. At least in theory.
Ironically, Java garbage collection seems to work too well, creating and removing too many objects. Most memory-management issues are solved, but often at the cost of creating serious performance problems. Making garbage collection adaptable to all kinds of situations has led to a complex and hard-to-optimize system. In order to wrap your head around garbage collection, you need first to understand how memory management works in a Java Virtual Machine (JVM).

How Garbage Collection Really Works

Many people think garbage collection collects and discards dead objects. In reality, Java garbage collection is doing the opposite! Live objects are tracked and everything else designated garbage. As you’ll see, this fundamental misunderstanding can lead to many performance problems.
Let’s start with the heap, which is the area of memory used for dynamic allocation. In most configurations the operating system allocates the heap in advance to be managed by the JVM while the program is running. This has a couple of important ramifications:
  • Object creation is faster because global synchronization with the operating system is not needed for every single object. An allocation simply claims some portion of a memory array and moves the offset pointer forward (see Figure 2.1). The next allocation starts at this offset and claims the next portion of the array.
  • When an object is no longer used, the garbage collector reclaims the underlying memory and reuses it for future object allocation. This means there is no explicit deletion and no memory is given back to the operating system.
New objects are simply allocated at the end of the used heap
Figure 2.1: New objects are simply allocated at the end of the used heap.
All objects are allocated on the heap area managed by the JVM. Every item that the developer uses is treated this way, including class objects, static variables, and even the code itself. As long as an object is being referenced, the JVM considers it alive. Once an object is no longer referenced and therefore is not reachable by the application code, the garbage collector removes it and reclaims the unused memory. As simple as this sounds, it raises a question: what is the first reference in the tree?

Garbage-Collection Roots—The Source of All Object Trees

Every object tree must have one or more root objects. As long as the application can reach those roots, the whole tree is reachable. But when are those root objects considered reachable? Special objects called garbage-collection roots (GC roots; see Figure 2.2) are always reachable and so is any object that has a garbage-collection root at its own root.
There are four kinds of GC roots in Java:
  • Local variables are kept alive by the stack of a thread. This is not a real object virtual reference and thus is not visible. For all intents and purposes, local variables are GC roots.
  • Active Java threads are always considered live objects and are therefore GC roots. This is especially important for thread local variables.
  • Static variables are referenced by their classes. This fact makes them de facto GC roots. Classes themselves can be garbage-collected, which would remove all referenced static variables. This is of special importance when we use application servers, OSGi containers or class loaders in general. We will discuss the related problems in the Problem Patterns section.
  • JNI References are Java objects that the native code has created as part of a JNI call. Objects thus created are treated specially because the JVM does not know if it is being referenced by the native code or not. Such objects represent a very special form of GC root, which we will examine in more detail in the Problem Patterns section below.
GC Roots are objects that are themselves referenced by the JVM and thus keep every other object from being garbage collected.
Figure 2.2: GC roots are objects that are themselves referenced by the JVM and thus keep every other object from being garbage-collected.
Therefore, a simple Java application has the following GC roots:
  • Local variables in the main method
  • The main thread
  • Static variables of the main class

Marking and Sweeping Away Garbage

To determine which objects are no longer in use, the JVM intermittently runs what is very aptly called a mark-and-sweep algorithm. As you might intuit, it’s a straightforward, two-step process:
  1. The algorithm traverses all object references, starting with the GC roots, and marks every object found as alive.
  2. All of the heap memory that is not occupied by marked objects is reclaimed. It is simply marked as free, essentially swept free of unused objects.
Garbage collection is intended to remove the cause for classic memory leaks: unreachable-but-not-deleted objects in memory. However, this works only for memory leaks in the original sense. It’s possible to have unused objects that are still reachable by an application because the developer simply forgot to dereference them. Such objects cannot be garbage-collected. Even worse, such a logical memory leak cannot be detected by any software (see Figure 2.3). Even the best analysis software can only highlight suspicious objects. We will examine memory leak analysis in the Analyzing the Performance Impact of Memory Utilization and Garbage Collection section, below.
When objects are no longer referenced directly or indirectly by a GC root, they will be removed. There are no classic memory leaks. Analysis cannot really identify memory leaks, it can only hint at suspicious objects
Figure 2.3: When objects are no longer referenced directly or indirectly by a GC root, they will be removed. There are no classic memory leaks. Analysis cannot really identify memory leaks; it can only point out suspicious objects.
Reference: http://www.journaldev.com/2856/java-jvm-memory-model-memory-management-in-java

Thursday, February 2, 2017

Compile Time Constants in Java

Compile Time Constants in Java

In this tutorial we’ll get a basic understanding of Compile Time Constants in Java programming language. This is not an exhaustive tutorial about compile time constants, but this should be enough for most people. We’ll understand what compile time constants are and how are they special. First lets go through the general rules regarding compile time constants.
  • They must be declared final
  • They are of primitive data types or String
  • They must be initialized with their declaration.
  • Their value must be constant expression.
If you don’t understand all of this, don’t worry, we’ll go through these rules now.
Final variables of primitive data types and Strings can be compile time constants. No other type of variables are compile time constants, not even the wrapper classes. They must also be initialized with their declaration, otherwise they’ll not be compile time constants. Lets take a few examples. All of the following are compile time constants,
final int i = 10;
final int j = 20, k = 30;
final String s = "Hello";
final float f = 10.5f;
The following are not compile time constants,
final Integer i = 10;    //not a primitive or String
int j = 10;              //not final
final int k;             //not initialized with declaration
= 10;
final int l, m = l = 20; //both l and m are not compile time constants
The last example might not be intuitive. We are initializing l and m in the same statement as their declaration, but they will not be compile time constants. Compile time constants have to be initialized right with their declaration and you can’t use a different variable in the middle of their declaration.
The last rule about compile time constants is that the compiler must be able to deduce their value. So their value can be any expression which contains literals and other compile time constants. Lets have a look at some valid values for compile time constants,
final int i = 10 * 20;  //uses only literals
final int j = i;        //uses another compile time constant
final int k = i * 20;   //mixing compile time constant and literal
If you use a variable in the expression to assign value to a final variable, then the compiler won’t be able to figure out the value of the final variable, so it will not be a compile time constant. The final variables given below are not compile time constants, because their value is not a constant expression,
int i = 10;
final int j = i;                 //using a variable
final int k = Math.round(10.2);  //method call


Compile Time Constants in Java! What’s So special About Them?

Now that we know what compile time constants are, we’ll now see why are they special. As the name implies, compile time constants get special treatment by the compiler. The first example of the special treatment is that they are implicit downcasted (just like literals). If you assign an int variable with value 10 to a short variable, you’ll get an error, but if you assign a int which is a compile time constant with the value 10 to a short variable, that will compile fine. Here is an example,
int a = 10;
short s1 = a;   //error
final int b = 10;
short s2 = b;   //fine
Since the compiler knows the value of compile time constants, the compler knows that the value of b is 10, so the compiler knows that the value of b is within the range of short data type. You can also use compile time constants as case values in switch-case statements as shown in the following example,
final int a = 10;
int b = 10;
switch(b) {
    case a:
        //some code
    case 20:
        //some code
}
But remember, case values cannot be String or floating point numbers, so you can’t use compile time constants of float, double or String data types as case values.
Both the behaviors that we observed above i.e. implicit downcast and case labels are because of a common reason. Wherever you use a compile time constant, the compiler replaces their use with their actual value. So if we write this code,
final int i = 10;
short j = i;
System.out.println(i);
After compilation it will become,
final int i = 10;
short j = 10;System.out.println(10);
As you can see, at both the places where we used i, the compiler replaced it with its value 10. This is the reason why they can be used as case values and that is why they are implicitly downcasted. This replacing of value has serious implications in some cases.
Suppose we have two classes in two different files like this,
public class ConstantClass {
    public static final int MY_CONST = 10;
}
public class Main {
    public static void main(String[] args) {
        System.out.println(ConstantClass.MY_CONST);
    }
}
Now if we compile both classes and run Main class, the output will be 10. Since MY_CONST is a compile time constant, so the compiler replaces it with its value in the main method. Now we go and change the value of MY_CONST to 20 and compile only ConstantClass class. Then we run the Main class (without compiling it again), the output will still be 10 i.e. the value of MY_CONST when Main class was compiled. We’ll have to recompile Main class to see the new value of MY_CONSTas the output.
One more special treatment that compile time constants receive is that you can declare static compile time constants inside non-static inner classes. As you might know, non-static inner classes cannot have static members. But you are allowed to add static compile time constants to non-static inner classes.


Wednesday, July 27, 2016

Try out with Swagger and Jersey Rest

Recently I had to try out few tools that can generate richer documentation for a Rest API.
I have created a basic demo project that shows off many features of Swagger.
I'm using Tomcat 7.0 , jdk 7.0 , Jersey 2.0 , Swagger 1.5x and eclipse as editor.

Follow the steps and also you can copy the project from my github project.

Step 1: Dependencies 

               <dependency>
   <groupId>javax.servlet</groupId>
   <artifactId>javax.servlet-api</artifactId>
   <version>3.1.0</version>
  </dependency>


  <!-- JAX-RS -->
  <dependency>
   <groupId>javax.ws.rs</groupId>
   <artifactId>javax.ws.rs-api</artifactId>
   <version>${jaxrs.version}</version>
  </dependency>
  <!-- Jersey 2.19 -->
  <dependency>
   <groupId>org.glassfish.jersey.containers</groupId>
   <artifactId>jersey-container-servlet</artifactId>
   <version>${jersey2.version}</version>
  </dependency>
  <dependency>
   <groupId>org.glassfish.jersey.core</groupId>
   <artifactId>jersey-server</artifactId>
   <version>${jersey2.version}</version>
  </dependency>
  <dependency>
   <groupId>org.glassfish.jersey.core</groupId>
   <artifactId>jersey-client</artifactId>
   <version>${jersey2.version}</version>
  </dependency>
  <dependency>
   <groupId>org.glassfish.jersey.media</groupId>
   <artifactId>jersey-media-json-jackson</artifactId>
   <version>${jersey2.version}</version>
  </dependency>

  <!-- Swagger dependencies -->
  <dependency>
   <groupId>io.swagger</groupId>
   <artifactId>swagger-jersey2-jaxrs</artifactId>
   <version>1.5.0</version>
  </dependency>

Step 2: Web.xml config

                <init-param>
   <param-name>jersey.config.server.provider.packages</param-name>
   <param-value>com.kant.rest,io.swagger.jaxrs.listing</param-value>
  </init-param>



Bootstrap class added to configure Swagger
        <servlet>
  <servlet-name>SwaggerBootstrap</servlet-name>
  <servlet-class>com.kant.rest.servlet.Bootstrap</servlet-class>
  <load-on-startup>2</load-on-startup>
 </servlet>
Filter to enable CORS support for tomcat
 <filter>
  <filter-name>CorsFilter</filter-name>
  <filter-class>org.apache.catalina.filters.CorsFilter</filter-class>
 </filter>
 <filter-mapping>
  <filter-name>CorsFilter</filter-name>
  <url-pattern>/*</url-pattern>
 </filter-mapping>

Step 3: Use of Swagger Annotations

/**Service endpoint**/
@Api(value = "hello", description = "Endpoint for user management")
@Path("/hello")
public class HelloWorldService {

 @GET
 @Path("/{param}")
 @Produces(MediaType.TEXT_PLAIN)
 @ApiOperation(value = "Returns welcome message.", notes = "Returns a  welcome message for the param name passed.", response = String.class)
 @ApiResponses(value = {
   @ApiResponse(code = 200, message = "Successful retrieval of response", response = String.class),
   @ApiResponse(code = 404, message = "Param passed is not good"),
   @ApiResponse(code = 500, message = "Internal server error") })
 public Response getMsg(@PathParam("param") String msg) {
  if ("bad".equalsIgnoreCase(msg)) {
   return Response.status(Status.BAD_REQUEST).build();
  }
  String output = "Jersey say : " + msg;
  return Response.status(200).entity(output).build();
 }

 @GET
 @Path("/person/{id}")
 @Produces(MediaType.APPLICATION_JSON)
 @ApiOperation(value = "Returns Person.", notes = "Returns a person details for value passed.", response = Person.class)
 @ApiResponses(value = {
   @ApiResponse(code = 200, message = "Successful retrieval of response", response = Person.class),
   @ApiResponse(code = 404, message = "Param passed is not good"),
   @ApiResponse(code = 500, message = "Internal server error") })
 public Response getPerson(@PathParam("id") String id) {
  Person person = new Person();
  if ("1".equals(id)) {
   person.setAge(27);
   person.setEmailId("itsmeshashi@outlook.com");
   person.setFirstName("shashi");
   person.setLastName("kant");
   person.setPhone("532600021");

  } else if (Integer.parseInt(id) < 5) {
   person.setAge(22);
   person.setEmailId("dummayMail@fake.com");
   person.setFirstName("foo");
   person.setLastName("bar");
   person.setPhone("2343-243-234");
  } else
   return Response.status(Status.BAD_REQUEST).build();
  return Response.status(200).entity(person).build();
 }

}

/**Model class**/
@ApiModel(value = "Person", description = "stores person details")
@XmlRootElement
public class Person {
 @XmlElement
 @ApiModelProperty(position = 1, required = true, value = "user's firstname containing only lowercase letters")
 private String firstName;
 @XmlElement
 @ApiModelProperty(position = 2, required = true, value = "user's last name containing only lowercase letters")
 private String lastName;
 @XmlElement
 @ApiModelProperty(position = 3, required = true, value = "email id ")
 private String emailId;
 @XmlElement
 @ApiModelProperty(position = 4, required = true, value = "phone number")
 private String phone;
 @XmlElement
 @ApiModelProperty(position = 5, required = true, value = "user's age")
 private Integer age;

 public String getFirstName() {
  return firstName;
 }

 public void setFirstName(String firstName) {
  this.firstName = firstName;
 }

 public String getLastName() {
  return lastName;
 }

 public void setLastName(String lastName) {
  this.lastName = lastName;
 }

 public String getEmailId() {
  return emailId;
 }

 public void setEmailId(String emailId) {
  this.emailId = emailId;
 }

 public String getPhone() {
  return phone;
 }

 public void setPhone(String phone) {
  this.phone = phone;
 }

 public Integer getAge() {
  return age;
 }

 public void setAge(Integer age) {
  this.age = age;
 }

}


Screenshots:





Notes:

1. Swagger-ui : copy zipped version, unzip and  open ./dist/index.html to launch Swagger UI in a browser. Now Explore your local api. Make sure CORS filter is added to your web.xml according to the server.
2. Swagger editor : try out live demo version and copy your swagger.json to test the editor right away
3. Other Swagger tools .


References:

1. https://github.com/swagger-api/swagger-core/wiki/Swagger-Core-Jersey-2.X-Project-Setup-1.5
2. http://stackoverflow.com/questions/16296145/set-cors-header-in-tomcat/18850438#18850438 

Project Page: https://github.com/thekant/jaxrs_swagger_rest_example




Wednesday, March 30, 2016

Statistics : calculating mean median mode and standard deviation

/**
 * 
 */
package com.kant.hackerrank.statistics;

import java.util.ArrayList;
import java.util.HashMap;
import java.util.Iterator;
import java.util.List;
import java.util.Map;
import java.util.Scanner;
import java.util.TreeMap;

/**
 * @author shaskant
 *
 */
public class Challenge1 {

 static double mean = 0;
 static double median = 0;
 static int mode = Integer.MAX_VALUE;
 static double deviationStd = 0;

 /**
  * @param args
  */
 public static void main(String[] args) {
  @SuppressWarnings("resource")
  Scanner scanner = new Scanner(System.in);
  int inputSize = scanner.nextInt();
  Map<Integer, Integer> store = new TreeMap<Integer, Integer>();
  int count = 0;
  for (int inputC = 0; inputC < inputSize; inputC++) {
   int inputN = scanner.nextInt();
   mean += inputN;

   if (store.containsKey(inputN)) {
    count = store.get(inputN).intValue();
    store.put(inputN, count + 1);
   } else {
    store.put(inputN, 1);
   }
  }
  mean = mean / inputSize;
  System.out.printf("%.1f\n", mean);
  calculateModeAndDeviation(store, inputSize);
  System.out.printf("%.1f\n", median);
  System.out.println(mode);
  System.out.printf("%.1f\n", deviationStd);
 }

 /**
  * 
  * @param store
  */
 private static void calculateModeAndDeviation(Map<Integer, Integer> store,
   int inputSize) {
  Map<Integer, List<Integer>> map = new HashMap<Integer, List<Integer>>();
  Iterator<Integer> iterator = store.keySet().iterator();
  int maxKey = Integer.MIN_VALUE;
  boolean isOdd = inputSize % 2 != 0;
  int inputC = 0;
  int[] data = new int[inputSize];
  while (iterator.hasNext()) {
   int next = iterator.next().intValue();
   Integer nextVal = store.get(next);
   deviationStd += (double) nextVal * Math.pow(next - mean, 2);
   if (map.containsKey(nextVal)) {
    map.get(nextVal).add(next);
   } else {
    List<Integer> list = new ArrayList<Integer>();
    list.add(next);
    map.put(nextVal, list);
   }
   if (maxKey < nextVal) {
    maxKey = nextVal;
   }
   for (int i = inputC; i < inputC + nextVal; i++)
    data[i] = next;
   inputC += nextVal;
  }

  if (!isOdd) {
   median = (double)(data[inputSize / 2] + data[inputSize / 2 - 1])/2;
  } else {
   median = data[inputSize / 2];
  }

  Iterator<Integer> iterator2 = map.get(maxKey).iterator();
  while (iterator2.hasNext()) {
   int next = iterator2.next().intValue();
   if (mode > next)
    mode = next;
  }

  deviationStd = Math.sqrt(deviationStd / inputSize);

 }
}