Hadoop, Spark & Big Data Blog Posts

Last updated: Mar 28, 2020

I try to post a fair amount to my Professional Blog on this site around Big Data technologies; just look for content with the "hadoop", "spark" and/or "big_data" label as shown below. Feel free to offer up thoughts on what my Upcoming Blog Posts should be about.

Blog:
topology supervision features of streaming frameworks (or lack thereof)
Blog:
are partially-written hdfs files accessible? (not exactly, but much more yes than I previously thought)
Blog:
use spark to calculate salary statistics for georgia educators (the fourth book of the trilogy)
Blog:
building a c# storm topology (yes, it is a jvm-based framework)
Blog:
learning something new every day (seems hdfs is not as immutable as i thought)
Blog:
accepting "best answer" on hcc (it is the ~right~ thing to do)
Blog:
NoClassDefFoundError for Log4jLoggerFactory on hdp 2.5.3 when running the KafkaSpout in your topology? (how's that for a title?)
Blog:
initial hbase grants on a new secure hadoop cluster (without ranger)
Blog:
my talk at devnexus (links to video and preso)
Blog:
why you should be a supporting "member" of oss (hey, it works for npr)
Blog:
joining multiple datasets with pig (i/o courtesy of hcatloader & hcatstorer)
Blog:
storing dynamically created file names with pig (piggbank's multistorage to the rescue)
Blog:
why spark's mapPartitions transformation is faster than map (calls your function once/partition, not once/element)
Blog:
performing a non-root ambari install (with hortonworks admin 1 course)
Blog:
need an overview of hadoop? (i need some reviewers)
Blog:
A Lightning Fast Trip to Johannesburg
Blog:
transitioned to training (and loving it)
Blog:
hadoop mount points (more art than science)
Blog:
trying out hive testbench on hdp sandbox (and packaging it up for deployment elsewhere)
Blog:
installing hdp 2.2 with ambari 2.0 (moving to the amazon cloud)
Blog:
presenting at hadoop summit (archiving evolving databases in hive)
Blog:
installing hdp 2.2 with ambari 2.0 (moving to the azure cloud)
Blog:
connecting dbvisualizer to hive (running on hdp 2.2)
Blog:
got a hadoop question? (then ask lester!)
- hadoop
- advice
Blog:
took the pig/hive test (got a shiny new certificate)
Blog:
os patching your hadoop cluster (pre & post rolling upgrades)
Blog:
parameterizing mapred.* properties (cli vs oozie)
Blog:
hadoop mini smoke test (VERY mini)
Blog:
help me go to belgium (not asking for money, just a couple of votes)
Blog:
improving datanode resiliency (it's all about the settings)
Blog:
simple hadoop cluster user provisioning process (simple = w/o pam or kerberos)
Blog:
hadoop worker node configuration recommendations (1-2-10)
Blog:
a patent for the "idea" of ingesting data into hadoop (is it really "sponge worthy"?)
Blog:
changes to hive's decimal datatype (it could cost you lots of pennies)
Blog:
stinger.next to the rescue (but you do have stinger.NOW tuning options available, well, "now")
Blog:
a robust set of hadoop master nodes (it is hard to swing it with two machines)
Blog:
obtained hortonworks' apache hadoop administrator certification (finally)
Blog:
hadoop security (it's all about layering)
Blog:
hadoop superuser (you can have more than 'hdfs')
Blog:
hadoop demystified presentation (with atlanta's .net user group)
Blog:
hadoop streaming with .net map reduce api (executing on hdp for windows)
Blog:
installing hdp on windows (and then running something on it)
Blog:
small files and hadoop's hdfs (bonus: an inode formula)
Blog:
setting up hdp 2.1 with non-standard users for hadoop services (why not use a non-standard user for ambari, too)
Blog:
using your mac to install a virtualized hadoop cluster? (then setup a local repo on it)
Blog:
use hive to calculate salary statistics for georgia educators (third of a three-part series)
Blog:
use pig to calculate salary statistics for georgia educators (second of a three-part series)
Blog:
use mapreduce to calculate salary statistics for georgia educators (first of a three-part series)
Blog:
hadoop component versions by distributions (the open source ones)
Blog:
feeling a bit prolific (or maybe i'm just a smart aleck)
Blog:
manually installing hue (on my virtualized 5-node cluster)
Blog:
create and share a hive udf (the cli is your friend)
Blog:
create and share a pig udf (anyone can do it)
Blog:
stopping oozie from limiting the number of reducers on your hive action (just add some more xml)
Blog:
how do i load a fixed-width formatted file into hive? (with a little help from pig)
Blog:
what's after the hortonworks sandbox? (a 5-node cluster!)
Blog:
building a virtualized 5-node HDP 2.0 cluster (all within a mac)
Blog:
disruptive possibilities (the rise of platform architecture)
Blog:
hadoop world 2013 (reflections from nyc)
Blog:
too big to ignore (too boring to read)
Blog:
fancy yourself a data scientist? (then show me the money!)
Blog:
hadoop yarn (in a nutshell)
Blog:
just reboot it (when was that ever a good idea?)
Blog:
i'm a certified hadoop developer (so, what does that mean?)
Blog:
what the world needs now is another nosql preso (like i need a hole in my head)