As part of my journey with Data Lake I have started to look at Python. Our road map includes mi grating to Data Bricks which makes use of the Python language for analyzing and accessing data.
Currently we have our we have a way of bringing data from a source system into the Data Lake that is configurable. We can add a record to a configuration database and data will be exported from the source system and stored on the Data Lake in the location that we have specified.
While working on our Data Lake implementation we ran into issues saving our data to CSV. The Extractor for CSV does not ignore line feeds inside data. Which was a huge problem that we had to handle. After not much conversation it was decided to use JSON now, we planned on phasing JSON in later.
Extraction of JSON files is not available “out of the box” with USQL, but there is a GitHub repostiory with a JSON Extractor that is highly recommended.
I am excited about a new project that I am involved in. We are moving our Business Intellignece System to a Data Lake. This has been in discussion for some time now, another team in our company is already using Azure Data Lake and having some great results. This project is challenging as well as fun, I am a tech guy…
I have learned so much during the planning of this project and we are ready to move to implementation.
UPDATED: March 10, 2019
I have had many different sites over the years.
Recently, a few months ago I decided to shut them down. I was not updating them as much and focusing to much on maintaining and updating the site. With family and work I am very busy like many of you. A friend introduced me to Wyam.io which is a static site generator and hosting using Azure Blob Storage.
I was able to attend CRMUG Summit again this year. The 2017 Summit was hosted in Nashville, TN at the Gaylord Opryland Resort. I have been so impressed with the amount of content and the people who attend Summit both times now.
My Goals Before I leave for the Summit, I always prepare by listing out my goals. The number of sessions available is amazing and it is very easy to become overwhelmed and miss some very good sessions.
As a technologist I have worked on several different types of teams over my 20+ years of technology development career. Also, as a person I have been involved in different types of groups of people that are working together to solve a common problem or outcome. Several years ago I began a journey with Agile and started working on a Scrum team.
I became a Certified Scrum Master and learned from some very intelligent and passionate people who know Agile and Scrum.
Last year I attended the CRMUG Summit in Tampa and have to say I was very impressed with everything that I was involved in. During the summit I attended two classes in the Black Belt Series and completed the series when I returned to Pittsburgh. The information that I learned in this series I put to immediate use on our Dynamics 365 project. I also met with a local Pittsburgh Chapter leader and discussed participating in the local chapter.
Last year it was 129 a day that we lost from overdose, this year it is 144 a day.
144 A Day We have to do more. We have to take action and speak out and help people. We have to find ways to help people even when they are NOT ready. We have to make them ready. I watched a video on Facebook this morning that really touches this point.
Everyone can make a difference. If you care about something, stand up, put yourself out there and do something.
The smallest thing can make a difference and if enough people do something small the change can be huge. We live in a world that is full of advancements and luxuries that where science fiction a few decades ago. Yet, we have a world that is so full of sadness and darkness.
I have been blogging since 2011! WOW!
Why do I blog? Technology Geek I work in technology and enjoy technology. I am amazed by the advancements that we are seeing on a rapid pace. I remember in the early 90s setting up internet “newsgroup” for a corporate BBS, before the web was what it is today. Now, people have access to so much information in the palms of their hands.
This year saw changes in my work life, I was contacted by a previous employer to come back as a principal engineer. The company and boss made this decision very easy, now that I am at a large international company some of my daily responsibilities have changed. But, all in all I am doing the same thing that I have been doing for that past 15+ years. I have been very involved with the social media communities and especially Pittsburgh based communities.
I have been keeping a journal for a long time. I find it very therapeutic to write my thoughts down and reflect on them. I started with pen and paper, writing entries in notebooks. The last few years I started to use Day One on my iPhone. I wanted to have a way of having my journal with me so I could record my thoughts whenever and wherever I was. I tried a few other applications on my phone, I really wanted something that I could use on my computer and iPhone.
My wife suggested that we pickup Apple Watches for ourselves. Originally we planned on waiting for the new Blaze by FitBit to be released. But, being the tech geek that I am, we thought let’s give it a try and if we don’t like it we can return it. It has been nearly a month now and I have to say I really like the Apple Watch a lot.
It is a very comfortable watch and doesn’t get in the way.
I started using Evernote back in 2011 and have found it to be so valuable to my everyday life. I use Evernote to log notes about my professional and personal life.
I keep my notes in Evernote organized with tags and notebooks. I have found it so easy to find information that I created. The ability to have my notes with me on my phone and computer made information easily available at anytime.
Scrum Master Rotation My team has implemented a Scrum Master rotation and just finished up the first 2 months of it. We had a retrospective dedicated just to this new process and had a lot of great feedback from it.
I found that it provided a whole different perspective on what we do and how we understand Scrum. But, I found that this also helped really protect the sprint with this rotation, we decided to make he rotation last 3 months and extended the current person another month.
Emily over at emilylevenson.com created a new Facebook group for Pittsburgh Bloggers, I joined the group right away. Those who do not know her, she also create Project: Food Budget which I have participate in for helping to manage my family’s food spending. I love the idea of a group for people who blog around Pittsburgh to communicate and get to know each other better. I am a new one to the blogging arena, I am not new to the web or Internet though.
This is a common error I have run across and can be solved by making a change on the client’s machine eConnect config file. This post is a reminder for me if I see this on a new image of a machine using eConnect.
Error message There was an error writing to the pipe: The pipe is being closed.` occurs on eConnect for Microsoft Dynamics GP 2010.
Add receiveTimeout=�infinite� to the Microsoft.
Well, as part of my cleanup and organization project with my domain names I decided to keep Rainaldi.org as my resume and technology blog. I will blog about things that I work with as well as new technologies.
Having been in the computer industry professionally for over 15 years now, I have been involved in projects covering various aspects of technology throughout several industries. I have benefited from this depth by gaining a strong knowledge of solution design and business practices from some of the best people in the industry and expanding my knowledge set to handling many different situations.
Part of the project I am on involves moving the entire WordPress site from our internal data center to an Amazon EC2 zone. Backup Buddy was recommended by a consultant that we are working with and I tested it against a Go Daddy account first.
Well, let’s same this is a fantastic plugin, I backed up the entire site (database and files) and moved the ZIP file as well as the importbuddy.
Well, let me start by saying that my background has always been in Microsoft technologies (VB, C#, ASP.NET SQL Server, etc). But, I have used others such as Java, Oracle, Sybase, PHP and MySQL; this has always been a strong point of my background that I can easily switch between environments.
A recent project at work pushed my hand to learn WordPress and like I usually do I jumped in full force.