Java Streams API in brief

First, let’s define what a stream is in Java 8: a sequence of functions, actions, inputs, and outputs (better defined as a “pipeline”). Streams API provides functional-style operations to transform these sequences; sources for them can contain arrays, collections, files, etc. In general terms, streams are Monads:

“Monads represent computations to be executed in a sequential (or parallel, in Java Streams) structure.”

Streams API is a great starting point for data preparation, and, if we compare it with its Python equivalents, Java provides us a very strong alternative for Data Science, as follows:

  • As a typed-language, the compiler can detect error and bugs.
  • Java bytecode is faster than scripting languages as R or Python
  • Versions of libraries in projects can be easily maintained with Maven or Gradle
  • Common big data frameworks such as Apache Hadoop or Spark are written in Java or JVM languages.
  • Creating models in Java makes easier to integrate them to production systems, which usually are written in Java or similar languages.

Let’s start some examples defining a simple Car class:

class Car {
     private final String name;
     private final Country origin;
 
     Car(String name, Country origin) {
         this.name = name;
         this.origin = origin;
     }
 
     public String getName() { return name; }
     public Country getOrigin() { return origin; }
 }
 
 final class Country {
     private final int value;
     private final String name;
 
     public static Country GERMANY = new Country(1, "GERMANY");
     public static Country US = new Country(2, "US");
     public static Country UK = new Country(3, "UK");
     public static Country INDIA = new Country(4, "INDIA");
     public static Country JAPAN = new Country(5, "JAPAN");
 
     private Country(int value, String name) {
         this.value = value;
         this.name = name;
     }
 
     public int getValue() { return value; }
     public String getName() { return name; }
 }

Streams API usually works with collections, so a useful method to convert an array into a collection is Arrays.stream:

Car[] cars = {
        new Car("GM", Country.US),
        new Car("Cadillac", Country.US),
        new Car("BMW", Country.GERMANY),
        new Car("Mercedes Benz", Country.GERMANY),
        new Car("Toyota", Country.JAPAN),
        new Car("Mazda", Country.JAPAN),
        new Car("Honda", Country.JAPAN),
        new Car("Mahindra", Country.INDIA),
        new Car("Land Rover", Country.UK)
};

List<Car> list = Arrays.asList(cars);
Stream<Car> stream = list.stream();

Streams are not reusable, so they have to be recreated in order to start a new processing pipeline, because of this I’m going to be using list.stream() in all the examples as follows:

List<String> germanCars = list.stream()
        .filter(x -> x.getOrigin().equals(Country.GERMANY))
        .map(Car::getName)
        .collect(Collectors.toList());

This piece of code shows some useful Stream functions:

1. Filtering.

It directly works on the stream, and receives a lambda function to evaluate the filter, it returns a filtered Stream

2. Mapping

It provides a useful way of selecting a specific data member of Car class, and “maps” the stream of Cars into a stream of Strings.

3. Collecting

It performs a data transformation into lists, sets, strings, etc, using Collectors class, which provides useful transformation methods, just like Collectors.toList()

Three more examples:
String rawSentence = list.stream()
        .map(Car::getName)
        .collect(Collectors.joining(","));
Set<String> countries = list.stream()
        .map(Car::getOrigin)
        .map(Country::getName)
        .collect(Collectors.toSet());
Map<Country, List<Car>> groupByCountry = list.stream()
        .collect(Collectors.groupingBy(Car::getOrigin));
System.out.println(groupByCountry.get(Country.JAPAN).stream()
        .map(Car::getName)
        .collect(Collectors.toList()));

In this last two examples, we can see that we can operate on a stream over and over again, and it creates the “pipelines” we mentioned before.

There is a useful toMap() collector that can index a collection using some fields. For example, if we want to get a map from Car names to Car objects, it can be achieved using the following code:

Map<String, Car> tokenToWord = list.stream()
        .collect(Collectors.toMap(Car::getName, Function.identity()));

Streams API provides streams of primitives (ints, doubles, and others), they have basic statistical methods such as sum, max, min, average, or summaryStatistics. For example:

int maxNameLength = list.stream()
        .mapToInt(x -> x.getName().length())
        .max().getAsInt();

Custom Collectors

We can also define our own collector by using Collector class which requires to pass a supplir class, accumulator, combiner and finisher method, for example:

Collector<Car, StringJoiner, String> carsCollector =
     Collector.of(
         () -> new StringJoiner(", "),
         (joiner, car) -> joiner.add(car.getName().toUpperCase()),
         StringJoiner::merge,
         StringJoiner::toString);
 
 String names = list.stream().collect(carsCollector);

4. Parallelizing

Streams can be execute in parallel taking advantage of the available physical CPU cores. Streams use a common ForkJoinPool, we can check the size of the underlying thread-pool by using ForkJoinPool.commonPool (7 threads in my PC). For example:

ForkJoinPool commonPool = ForkJoinPool.commonPool();
 System.out.println("Threads: " + commonPool.getParallelism());
 int[] firstLengths = list.parallelStream()
         .filter(w -> w.getOrigin().equals(Country.JAPAN))
         .map(Car::getName)
         .mapToInt(String::length)
         .sequential()
         .sorted()
         .limit(2)
         .toArray();
 System.out.println(Arrays.toString(firstLengths));

5. I/O

Finally, we can use Java I/O library, for example, using files to be represented as a stream of lines using Reader.lines() method.

InputStream fStream = Main.class.getResourceAsStream("text.txt");
 InputStreamReader fReader = new InputStreamReader(fStream, StandardCharsets.UTF_8);
 
 try (Stream<String> lines = new BufferedReader(fReader).lines()) {
     double average = lines
             .flatMap(line -> Arrays.stream(line.split(" ")))
             .map(String::toLowerCase)
             .mapToInt(String::length)
             .average().getAsDouble();
     System.out.println("average token length: " + average);
 }

Conclusion

Streams are a very powerful feature introduced in Java 8 (and to be extended soon in Java 9), very useful to process data for Data Science in Java. There are much more useful methods in Streams, which  More examples in the future.

Go to GitHub Repo: https://github.com/alulema/JavaDS.01

Is it worth it to work as a freelancer?

BACKGROUND

Usually many of us graduates from college have dreamed of working in a large company with great salary, prestige and traveling around the world; or maybe we’ve had the “million-dollar idea” and we’ve put tears, sweat and blood to boot that beloved startup and appear in Wired as the startup of the year. No doubt many have experienced one of these two scenarios, which is commendable; for other people perhaps this was not as important as it was to lead a quiet and familiar life with a modest and stable job without the need to experience the jetlag of trips at every moment, or the stressing deadlines for launching the product; those who have chosen this option have chosen the quiet life and are also congratulated.

But there are those who have found no satisfaction in either case, whether due to failures, lost opportunities or simply the opportunity was never presented . The stablishment may lead them to think of them as “loosers”, but nothing further from true, as they may be potential or emerging freelancers.

MY EXPERIENCE

Some years before starting my adventure as a freelancer I failed 2 times with a startup that never launched and succeded, and missed opportunities to work in spectacular companies. This left me with the thought that maybe that was not for me, and I decided to take a normal job and lead a quiet life. However, deep within me there was still a spark of entrepreneurship, and on the day my daughter was born, the call was reborn. Maybe it might seem like this was not the best time to do it, but there was now a need to organize my time much better so I could have hours of the day and see my daughter grow up.

Since then I have learned many things that I once considered obvious, but now I consider important in freelance work:

1. Enjoy the hours of the day

With this I don’t mean to work 20 hours a day, but it is important to give each task its time and space, and execute it in an excellent way: work well, get enough sleep, rest the necessary, a good time in family, a good time to leisure. The unsuccessful tasks are those that don’t contribute,

2. Dress for success

From experience I can say that your body “feels” that we are going to work and is predisposed to perform efficiently. Many people think that the freelancer works in pajamas, but doing so will simply predispose the body to being sleepy.

3. Do not neglect your work

At first we do not know the price we will give to our work, one tends to think that we could charge expensive and lose the customer, or very cheap and become a bargain. It may actually be a mid-point, but you need to consider that you probably have to cover the following costs:

  • Error correction
  • Delays in deadlines or payments
  • Unexpected expenses and delays

It is important to say that often the client is not responsible for these inconveniences, but the wise freelancer should be prevented. An approximate percentage is ~ 20% of the estimated time. If all goes well and there are no unexpected events, that surplus will be a backup until you get the next client or project.

4. The horse prepares for battle

The worst thing we can do is feel comfortable with what we learned the previous year, the skills required for the freelancer are always changing, and a continuous learning prepares us to be able to professionally assume each project.

5. Personal branding is vital

Personally I am not a fan of social networks, however, promoting our name is very important, mainly with the idea of ​​projecting us as professionals. The important resources to take in account are:

  • LinkedIn
  • Personal Site or Blog/Vlog
  • GitHub
  • Twitter

6. Search and you will find

Being persistant is extremely important, although you can wait for “capturing” a project in order to begin, once you start walking as a freelancer you will have to go looking for projects. Hopefully you will find a project right away, and you will have to enter to the field of being Project Manager and Developer at the same time; but usually starting a new project with the client takes a little time, in my experience about 2 months between starting the conversations and writing the first line of code

CONCLUSIONS

The adventure of being a freelancer is exciting because it involves 2 worlds: being an entrepreneur and being an employee, starting up involves a great effort until reaching the inertia and the necessary know-how for development. Are you wondering if it can be lucrative? Of course, it is lucrative enough not to envy employment in any large company, but it will require more time on your part. If you value family life and laisure times, it is possible to find a balance between incoming and free time to devote to other activities.

One very interesting option is being part of freelancing platforms like Toptal Web Engineering Group, which considers common needs of freelancers around the world. The defying task of getting clients is performed by TopTal, but it requires from you to be the best in what you do because only “la creme dela creme” of developers is chosen to work with TopTal. So, it’s hard but not impossible. Also, you can’t compare TopTal with other portals like Elance or Freelancers because they are too hard to get clients, even more complcated than finding projects by yourself in your local environment. I was able to work with Elance before, and it was good because of the client, but competence can be very hard and it’s not always a win-to-win relationship.

For those who intend to take this step of faith my recommendation is only to do it with determination and dedication. It may be very helpful to have an economic backup to get you started, but adventurers may enjoy cruising the desert to get to the promised land. Go ahead and Bon Voyage!

Cross-Platform Communications: gRPC Server and Client: .NET Core

When we are developing web solutions consisting of several projects, it is very common communicate between each other, and the common solutions are proprietary solutions (like .NET Remoting), or standard solutions like REST or SOAP. Recently, I discovered gRPC as the Google solution for Cross-Platform communications, allowing the developers to communicate applications. The advantage of gRPC is that it doesn’t use message formats which need to assemble JSON or XML (and it makes a bigger message, sometimes). Instead of this, gRPC uses Protocol Buffers to define binary messages to be interchanged between the parts.

At this time, I start with .NET Core, and you can see the details for creating server and client:

GitHub Repository: https://github.com/alexis-dotnet/Dotnet.Grpc

protoc command: protoc -I=pb –csharp_out=Messages pb/messages.proto –grpc_out=Messages –plugin=protoc-gen-grpc=/Users/_your username_/.nuget/packages/Grpc.Tools/1.1.0/tools/macosx_x64/grpc_csharp_plugin
chmod 755 generateCerts.sh

Links:
gRPC: http://www.grpc.io/
Protocol Buffers: https://developers.google.com/protocol-buffers/
SSH Keys Generation: http://stackoverflow.com/a/37739265

Installing Redis on Azure Ubuntu VM

In one of my latest developments I needed to have a local Redis for caching requirements. I feel comfortable on terminal/command line, so it wasn’t hard to have an Ubuntu VM on VirtualBox, and it is very helpful to have it running on cloud if you need a testing site; nevertheless, there are several commands you need to remember to install it properly. Because of this, I created the following video installing Redis on an Ubuntu VM on Microsoft Azure. Here the details:

Installing Redis on Azure Ubuntu VM (and getting connected from Windows)

1. Creating Ubuntu VM on Azure
2. SSH Connection to VM
3. Installing and configuring Redis
4. Creating rule for receiving connections in Azure
5. Connection from Windows Redis client

ASP.NET Core Basics – Part 2

Now, we continue with the series ASP.NET Core Basics. At this time we are been focused in MVC specific topics, which will be covered in a 7 mins video.

Contents:
1. MVC. Controllers
2. MVC. Models and Views
3. ViewModel
4. Post/Redirect/Get
5. Model Validation

Links:

MVC Pattern: https://en.wikipedia.org/wiki/Model–view–controller
Post/Redirect/Get Pattern: https://en.wikipedia.org/wiki/Post/Redirect/Get
Github Repository: https://github.com/alexis-dotnet/NewsApp_Stage2
The New York Times API: https://developer.nytimes.com/

Español: Conceptos Básicos ASP.NET Core P.2
Português: Conceitos Básicos ASP.NET Core P.2

Installing SQL Server for Linux on Docker Container

It is very common requiring MS SqlServer if you are developing ASP.NET web applications; because of this, having a local Sql Server is a great plus if you are developing web apps on ASP.NET Core on Mac, or Linux. One of the things to take in consideration is that we can make a “clean and rapid installation” using a Docker container. Take a look at this video for instructions!

1. Installing Docker
2. Installing Kitematic
3. Installing and using SQL Server Client (sql-cli)
4. Installing mssql extension on VSCode
5. Configuring connection on VSCode

ASP.NET Core Basics – Part 1

I’m starting this blog with a series of videos focused on ASP.NET Core. This tutorial has been completely developed on C# + Visual Studio Code + Mac OSX, but (almost) all commands and concepts are fully compatible with Windows and Linux.

Contents:
1. Project Startup
2. Configuration and Services
3. Startup and Middleware

Links:

Github Repo: https://github.com/alexis-dotnet/NewApp_Stage1
Microsoft .NET Core: https://www.microsoft.com/net/core
Node JS: https://nodejs.org/en/
Yeoman: http://yeoman.io/
Visual Studio Code: https://code.visualstudio.com/

Español: Conceptos Básicos ASP.NET Core P.1

Português: Conceitos Básicos ASP.NET Core P.1

Blog at WordPress.com.

Up ↑