Lesson 1: Intro to plotting data in R with ggplot
Functions for Lesson 1
?
, str
, glimpse
, summary
, table
, min
, max
, ggplot
, geom_point
, geom_smooth
, theme_minimal
, theme_classic
, theme_tufte
Packages for Lesson 1
tidyverse
, ggplot2
, dplyr
Agenda
Data visualisation in R
for Data Science, Section 3.1.1.
- Intro to the
R
environment (IDE)
- Loading packages, e.g.
tidyverse
- Using built-in
R
data: the mpg
dataset
- Using ggplot with the built-in data set (to make scatterplots)
- Modifying plot aesthetics
- Reading in outside data: Airbnb data
- Plotting Airbnb data with ggplot
Intro to the R
environment (IDE)
The RStudio
integrated development environment (IDE) and what you can do with it.
A more complete example of what you can acheive with the interface.
Loading packages, e.g. tidyverse
How to load packages in R
.
install.packages("tidyverse") # install package
library(tidyverse) # load the package library
require(tidyverse) # same as library
# We are typing in an R Script. Things with # in front make them comments and notes to ourselves
# Command Return to execute the line/ 'run the code'
Using built-in R
data: the mpg
dataset
Section 3.2.1
We'll use a built-in tidyverse dataset called mpg
with data about cars and gas-mileage.
mpg
# run help page with '?'
`?`(mpg)
- This is a tibble (data frame) that we've "printed" out. It's like R's version of an excel spreadsheet, but much better.
- A tibble will show us the first 10 rows, rows containing the data, column names, and the class of data within each column, such as numeric, integer, or character.
Summarising data
str(mpg) # structure of data
glimpse(mpg) # preview of data
summary(mpg) # basic summary stats
table(mpg$manufacturer) # counts of each column
head(mpg) # visualise first 6 rows of data
tail(mpg, 10) # visualise last 10 (or N) rows of data
names(mpg) # get column names
class(mpg) # class of data frame
class(mpg$manufacturer) # class of data column
mpg$displ # print a column
mpg$hwy # print a column
Creating a plot with ggplot
Section 3.2.2
ggplot()
Creates a coordinate system for us--basically an empty graph.
geom_point()
Adds a "layer", e.g. geom_point (but there are many for different kinds of graphs).
Plot two of the data columns
ggplot(data = mpg) + geom_point(mapping = aes(x = displ, y = hwy))
Changing the data column inputs for the x and y axis of the plot
ggplot(data = mpg) + geom_point(mapping = aes(x = class, y = drv))
Assign data to variables to create dynamic inputs
my_data <- mpg # create own variable using a name of your choice
ggplot(data = my_data) + geom_point(mapping = aes(x = displ, y = hwy))
Themes
Change plot style. Link for more ggplot themes.
require(ggthemes)
# classic theme
ggplot(data = my_data) + geom_point(mapping = aes(x = displ, y = hwy)) + theme_minimal()
# minimal theme
ggplot(data = my_data) + geom_point(mapping = aes(x = displ, y = hwy)) + theme_tufte()
# assign theme to variable
my_theme <- theme_classic()
ggplot(data = my_data) + geom_point(mapping = aes(x = displ, y = hwy)) + my_theme # apply your chosen theme
Aesthetic mapping
Section 3.3
colour
. Change the colour of the data points. size
. Change the size of the data points.
alpha
. Change the transparency of the data points.
Colour
Colour by colour name.
ggplot(data = my_data) + geom_point(mapping = aes(x = displ, y = hwy), colour = "light blue") + my_theme
Colour by a hex code in quotes.
ggplot(data = my_data) + geom_point(mapping = aes(x = displ, y = hwy), colour = "#BB5C42") + my_theme
colour by data column
ggplot(data = my_data) + geom_point(mapping = aes(x = displ, y = hwy, colour = class)) + my_theme
Inside versus outside the aes
ggplot(data = my_data) + geom_point(mapping = aes(x = displ, y = hwy, colour = "blue")) + my_theme
Size
Size by integer
ggplot(data = my_data) + geom_point(mapping = aes(x = displ, y = hwy, size = 5)) + my_theme
Size by data column
ggplot(data = my_data) + geom_point(mapping = aes(x = displ, y = hwy, size = class)) + my_theme
We get a warning, but this is okay.
Transparency
# map classe column to different transparencies
ggplot(data = my_data) + geom_point(mapping = aes(x = displ, y = hwy, alpha = class)) + my_theme
Shape
ggplot(data = my_data) + geom_point(mapping = aes(x = displ, y = hwy, shape = class)) + my_theme
Any warnings? Yes, because shape maxes out at six levels.
Manually changing aesthetic properties
But we can set the aesthetic properties manually, instead of having ggplot do the scaling automatically. For example, we can make our ggplot points all blue like this. This time, putting colour OUTSIDE the aes
argument.
ggplot(data = my_data) + geom_point(mapping = aes(x = displ, y = hwy), colour = "blue") + my_theme
Using colour both inside and outside the aes
ggplot(data = my_data) + geom_point(mapping = aes(x = displ, y = hwy, colour = class), colour = "#AE42BB") +
my_theme
The inner one is overridden.
Putting it all together as a snapshot of what's possible
ggplot(data = my_data) + geom_point(mapping = aes(x = displ, y = hwy, colour = class, size = class, alpha = class)) +
my_theme
Aesthetics you can manually set
- The name of a colour as a character string.
- The size of a point in mm.
- The shape of a point as a number, as shown in Figure 3.1.
R
has 25 built in shapes that are identified by numbers. There are some seeming duplicates: for example, 0, 15, and 22 are all squares. The difference comes from the interaction of the colour
and fill
aesthetics. The hollow shapes (0--14) have a border determined by colour
; the solid shapes (15--18) are filled with colour
; the filled shapes (21--24) have a border of colour
and are filled with fill
.
Further plotting examples
Section 3.3.1
The online reference contains further examples of how to visualise your data.
Reading in outside data: NYC Airbnb data
# A tibble: 6 x 74
id listing_url scrape_id last_scraped name description neighborhood_ov… picture_url host_id
<dbl> <chr> <dbl> <date> <chr> <chr> <chr> <chr> <dbl>
1 2595 https://www… 2.02e13 2021-04-09 Skyli… "Beautiful,… Centrally locat… https://a0… 2845
2 3831 https://www… 2.02e13 2021-04-12 Whole… "Enjoy 500 … Just the right … https://a0… 4869
3 5121 https://www… 2.02e13 2021-04-09 Bliss… "<b>The spa… <NA> https://a0… 7356
4 5136 https://www… 2.02e13 2021-04-10 Spaci… "We welcome… <NA> https://a0… 7378
5 5178 https://www… 2.02e13 2021-04-12 Large… "Please don… Theater distric… https://a0… 8967
6 5203 https://www… 2.02e13 2021-04-09 Cozy … "Our best g… Our neighborhoo… https://a0… 7490
# … with 65 more variables: host_url <chr>, host_name <chr>, host_since <date>,
# host_location <chr>, host_about <chr>, host_response_time <chr>, host_response_rate <chr>,
# host_acceptance_rate <chr>, host_is_superhost <lgl>, host_thumbnail_url <chr>,
# host_picture_url <chr>, host_neighbourhood <chr>, host_listings_count <dbl>,
# host_total_listings_count <dbl>, host_verifications <chr>, host_has_profile_pic <lgl>,
# host_identity_verified <lgl>, neighbourhood <chr>, neighbourhood_cleansed <chr>,
# neighbourhood_group_cleansed <chr>, latitude <dbl>, longitude <dbl>, property_type <chr>,
# room_type <chr>, accommodates <dbl>, bathrooms <lgl>, bathrooms_text <chr>, bedrooms <dbl>,
# beds <dbl>, amenities <chr>, price <chr>, minimum_nights <dbl>, maximum_nights <dbl>,
# minimum_minimum_nights <dbl>, maximum_minimum_nights <dbl>, minimum_maximum_nights <dbl>,
# maximum_maximum_nights <dbl>, minimum_nights_avg_ntm <dbl>, maximum_nights_avg_ntm <dbl>,
# calendar_updated <lgl>, has_availability <lgl>, availability_30 <dbl>, availability_60 <dbl>,
# availability_90 <dbl>, availability_365 <dbl>, calendar_last_scraped <date>,
# number_of_reviews <dbl>, number_of_reviews_ltm <dbl>, number_of_reviews_l30d <dbl>,
# first_review <date>, last_review <date>, review_scores_rating <dbl>,
# review_scores_accuracy <dbl>, review_scores_cleanliness <dbl>, review_scores_checkin <dbl>,
# review_scores_communication <dbl>, review_scores_location <dbl>, review_scores_value <dbl>,
# license <lgl>, instant_bookable <lgl>, calculated_host_listings_count <dbl>,
# calculated_host_listings_count_entire_homes <dbl>,
# calculated_host_listings_count_private_rooms <dbl>,
# calculated_host_listings_count_shared_rooms <dbl>, reviews_per_month <dbl>
Using a smaller dataset
# smaller csv file (16 cols)
url <- "http://data.insideairbnb.com/united-states/ny/new-york-city/2021-04-07/data/listings.csv.gz"
nyc <- read_csv(url)
nyc <- nyc[nyc$id < 20000, ] # get smaller subet of data
length(nyc$id) # print length of 'id' column
head(nyc)
Plotting AirBnB data with ggplot
Using the above plotting functions to visualise the AirBnB data
# plot neighborhood_group vs price
ggplot(data = nyc) + geom_point(mapping = aes(x = neighbourhood_group_cleansed, y = price, colour = neighbourhood_group_cleansed),
shape = 21, stroke = 1) + my_theme
# plot minimum_nights vs price
ggplot(data = nyc) + geom_point(mapping = aes(x = minimum_nights, y = price, colour = neighbourhood_group_cleansed),
shape = 20, size = 3, stroke = 1) + my_theme
# availability_365 vs price
ggplot(data = nyc) + geom_point(mapping = aes(x = availability_365, y = price, colour = neighbourhood_group_cleansed),
shape = 21, stroke = 1) + my_theme
# plot longitude vs price
ggplot(data = nyc) + geom_point(mapping = aes(x = longitude, y = price, colour = neighbourhood_group_cleansed),
shape = 21, stroke = 1) + my_theme
Try your own plot using the other variables in the dataset
# plot neighborhood_group vs price
names(airbnb)
glimpse(airbnb)
my_data <- NULL
x <- NULL
y <- NULL
colour <- NULL
shape <- NULL
stroke <- NULL
LS0tCnBhcmFtczoKICBsZXNzb246ICJMZXNzb24gMSIKICB0aXRsZTogIkludHJvIHRvIHBsb3R0aW5nIGRhdGEgaW4gUiB3aXRoIGBnZ3Bsb3RgIgogIGJvb2tjaGFwdGVyX25hbWU6ICJTZWN0aW9uIDMuMS4xIiAgICAKICBib29rY2hhcHRlcl9zZWN0aW9uOiAiaHR0cHM6Ly9yNGRzLmhhZC5jby5uei9kYXRhLXZpc3VhbGlzYXRpb24uaHRtbCIKICBmdW5jdGlvbnM6ICJgP2AsIGBzdHJgLCBgZ2xpbXBzZWAsIGBzdW1tYXJ5YCwgYHRhYmxlYCwgYG1pbmAsIGBtYXhgLCBgZ2dwbG90YCwgYGdlb21fcG9pbnRgLCBgZ2VvbV9zbW9vdGhgLCBgdGhlbWVfbWluaW1hbGAsIGB0aGVtZV9jbGFzc2ljYCwgYHRoZW1lX3R1ZnRlYCIKICBwYWNrYWdlczogImB0aWR5dmVyc2VgLCBgZ2dwbG90MmAsIGBkcGx5cmAiICAgICAgCiAgIyBlbmQgaW5wdXRzIC0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLQpoZWFkZXItaW5jbHVkZXM6IFx1c2VwYWNrYWdle2Zsb2F0fQphbHdheXNfYWxsb3dfaHRtbDogeWVzCm91dHB1dDoKICBodG1sX2RvY3VtZW50OgogICAgY29kZV9mb2xkaW5nOiBzaG93CiAgICBjb2RlX2Rvd25sb2FkOiB0cnVlCi0tLQogIApgYGB7ciwgc2V0dXAsIGVjaG8gPSBGQUxTRSwgY2FjaGUgPSBGQUxTRSwgaW5jbHVkZSA9IEZBTFNFfQpvcHRpb25zKHdpZHRoPTEwMCkKa25pdHI6Om9wdHNfY2h1bmskc2V0KAogIGV2YWwgPSBGQUxTRSwgIyBydW4gYWxsIGNvZGUKICBlY2hvID0gVFJVRSwgIyBzaG93IGNvZGUgY2h1bmtzIGluIG91dHB1dCAKICB0aWR5ID0gVFJVRSwgIyBtYWtlIG91dHB1dCBhcyB0aWR5CiAgbWVzc2FnZSA9IEZBTFNFLCAgIyBtYXNrIGFsbCBtZXNzYWdlcwogIHdhcm5pbmcgPSBGQUxTRSwgIyBtYXNrIGFsbCB3YXJuaW5ncyAKICBjb21tZW50ID0gIiIsCiAgdGlkeS5vcHRzPWxpc3Qod2lkdGguY3V0b2ZmPTEwMCksICMgc2V0IHdpZHRoIG9mIGNvZGUgY2h1bmtzIGluIG91dHB1dAogIHNpemU9InNtYWxsIiAjIHNldCBjb2RlIGNodW5rIHNpemUKICApCmBgYApcCgo8IS0tIGluc3RhbGwgcGFja2FnZXMgLS0+CmBgYHtyLCBsb2FkIHBhY2thZ2VzLCBldmFsPVQsIGluY2x1ZGU9VCwgY2FjaGU9RiwgbWVzc2FnZT1GLCB3YXJuaW5nPUYsIHJlc3VsdHM9J2hpZGUnLGVjaG89Rn0KcGFja2FnZXMgPC0gYygiZ2dwbG90MiIsImdndGhlbWVzIiwiZHBseXIiLCJ0aWR5dmVyc2UiLCJ6b28iLCJSQ29sb3JCcmV3ZXIiLCJ2aXJpZGlzIiwicGx5ciIpICAgCmlmIChyZXF1aXJlKHBhY2thZ2VzKSkgewogICAgaW5zdGFsbC5wYWNrYWdlcyhwYWNrYWdlcyxkZXBlbmRlbmNpZXMgPSBUKQogICAgcmVxdWlyZShwYWNrYWdlcykKICAgICMgbG9hZCB0dnRoZW1lcwogICAgIyBkZXZ0b29sczo6aW5zdGFsbF9naXRodWIoIlJ5by1ONy90dnRoZW1lcyIpCn0KbGFwcGx5KHBhY2thZ2VzLGxpYnJhcnksY2hhcmFjdGVyLm9ubHk9VCkgIApgYGAKCjwhLS0gX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fXyAtLT4KPCEtLSBfX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fIC0tPgo8IS0tIF9fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX18gLS0+CjwhLS0gc3RhcnQgYm9keSAtLT4KCiMgYHIgcGFzdGUwKHBhcmFtcyRsZXNzb24sIjogIixwYXJhbXMkdGl0bGUpYCAgICAKXCAgCgpGdW5jdGlvbnMgZm9yIGByIHBhcmFtcyRsZXNzb25gICAKYHIgcGFyYW1zJGZ1bmN0aW9uc2AgICAgClwgICAgCgpQYWNrYWdlcyBmb3IgYHIgcGFyYW1zJGxlc3NvbmAgICAgICAgICAgCmByIHBhcmFtcyRwYWNrYWdlc2AgICAgICAgIApcICAgIAoKIyBBZ2VuZGEgIAoKW0RhdGEgdmlzdWFsaXNhdGlvbiBpbiBgUmAgZm9yIERhdGEgU2NpZW5jZSwgYHIgcGFyYW1zJGJvb2tjaGFwdGVyX25hbWVgXShgciBwYXJhbXMkYm9va2NoYXB0ZXJfc2VjdGlvbmApLiAgICAKCjwhLS0gIGVuZCB5YW1sIHRlbXBsYXRlLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLSAtLT4KCiogSW50cm8gdG8gdGhlIGBSYGVudmlyb25tZW50IChJREUpICAKKiBMb2FkaW5nIHBhY2thZ2VzLCBlLmcuIGB0aWR5dmVyc2VgICAKKiBVc2luZyBidWlsdC1pbiBgUmAgZGF0YTogdGhlIGBtcGdgIGRhdGFzZXQgICAKKiBVc2luZyBnZ3Bsb3Qgd2l0aCB0aGUgYnVpbHQtaW4gZGF0YSBzZXQgKHRvIG1ha2Ugc2NhdHRlcnBsb3RzKSAgICAKKiBNb2RpZnlpbmcgcGxvdCBhZXN0aGV0aWNzICAKKiBSZWFkaW5nIGluIG91dHNpZGUgZGF0YTogQWlyYm5iIGRhdGEgIAoqIFBsb3R0aW5nIEFpcmJuYiBkYXRhIHdpdGggZ2dwbG90ICAgICAgCgpcICAKCiMgSW50cm8gdG8gdGhlIGBSYCBlbnZpcm9ubWVudCAoSURFKSAgCgpUaGUgYFJTdHVkaW9gIGludGVncmF0ZWQgZGV2ZWxvcG1lbnQgZW52aXJvbm1lbnQgKElERSkgYW5kIHdoYXQgeW91IGNhbiBkbyB3aXRoIGl0LiAgClwgICAgCgo8IS0tIC0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tIGltYWdlIC0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLSAtLT4KPGRpdiBhbGlnbj0iY2VudGVyIj4KICA8aW1nIHNyYz0iaW1nL3JzdHVkaW8xLmpwZyIgc3R5bGU9d2lkdGg6MTAwJT4KPC9kaXY+CjwhLS0gLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0gaW1hZ2UgLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tIC0tPgpcICAKCkEgbW9yZSBjb21wbGV0ZSBleGFtcGxlIG9mIHdoYXQgeW91IGNhbiBhY2hlaXZlIHdpdGggdGhlIGludGVyZmFjZS4gIApcICAKCjwhLS0gLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0gaW1hZ2UgLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tIC0tPgo8ZGl2IGFsaWduPSJjZW50ZXIiPgogIDxpbWcgc3JjPSJpbWcvcnN0dWRpbzIuanBnIiBzdHlsZT13aWR0aDoxMDAlPgo8L2Rpdj4KPCEtLSAtLS0tLS0tLS0tLS0tLS0tLS0tLS0tLSBpbWFnZSAtLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0gLS0+ClwgIAoKIyBMb2FkaW5nIHBhY2thZ2VzLCBlLmcuIGB0aWR5dmVyc2VgICAKCkhvdyB0byBsb2FkIHBhY2thZ2VzIGluIGBSYC4gIAoKYGBge3J9Cmluc3RhbGwucGFja2FnZXMoInRpZHl2ZXJzZSIpICAjIGluc3RhbGwgcGFja2FnZQpsaWJyYXJ5KHRpZHl2ZXJzZSkgIyBsb2FkIHRoZSBwYWNrYWdlIGxpYnJhcnkKcmVxdWlyZSh0aWR5dmVyc2UpICMgc2FtZSBhcyBsaWJyYXJ5ICAgIAoKIyBXZSBhcmUgdHlwaW5nIGluIGFuIFIgU2NyaXB0LiBUaGluZ3Mgd2l0aCAjIGluIGZyb250IG1ha2UgdGhlbSBjb21tZW50cyBhbmQgbm90ZXMgdG8gb3Vyc2VsdmVzCiMgQ29tbWFuZCBSZXR1cm4gdG8gZXhlY3V0ZSB0aGUgbGluZS8gInJ1biB0aGUgY29kZSIKYGBgClwgIAoKIyBVc2luZyBidWlsdC1pbiBgUmAgZGF0YTogdGhlIGBtcGdgIGRhdGFzZXQgICAKW1NlY3Rpb24gMy4yLjFdKGh0dHBzOi8vcjRkcy5oYWQuY28ubnovZGF0YS12aXN1YWxpc2F0aW9uLmh0bWwjdGhlLW1wZy1kYXRhLWZyYW1lKSAgCgpXZSdsbCB1c2UgYSBidWlsdC1pbiB0aWR5dmVyc2UgZGF0YXNldCBjYWxsZWQgYG1wZ2Agd2l0aCBkYXRhIGFib3V0IGNhcnMgYW5kIGdhcy1taWxlYWdlLgoKYGBge3J9Cm1wZwojIHJ1biBoZWxwIHBhZ2Ugd2l0aCAnPycKP21wZyAgIApgYGAKCiogVGhpcyBpcyBhIHRpYmJsZSAoZGF0YSBmcmFtZSkgdGhhdCB3ZSd2ZSAicHJpbnRlZCIgb3V0LiBJdCdzIGxpa2UgUidzIHZlcnNpb24gb2YgYW4gZXhjZWwgc3ByZWFkc2hlZXQsIGJ1dCBtdWNoIGJldHRlci4gCiogQSB0aWJibGUgd2lsbCBzaG93IHVzIHRoZSBmaXJzdCAxMCByb3dzLCByb3dzIGNvbnRhaW5pbmcgdGhlIGRhdGEsIGNvbHVtbiBuYW1lcywgYW5kIHRoZSBjbGFzcyBvZiBkYXRhIHdpdGhpbiBlYWNoIGNvbHVtbiwgc3VjaCBhcyBudW1lcmljLCBpbnRlZ2VyLCBvciBjaGFyYWN0ZXIuICAgIAoKIyMgU3VtbWFyaXNpbmcgZGF0YSAgICAgICAgCmBgYHtyfQpzdHIobXBnKSAjIHN0cnVjdHVyZSBvZiBkYXRhCmdsaW1wc2UobXBnKSAjIHByZXZpZXcgb2YgZGF0YSAKc3VtbWFyeShtcGcpICMgYmFzaWMgc3VtbWFyeSBzdGF0cyAgCnRhYmxlKG1wZyRtYW51ZmFjdHVyZXIpICMgY291bnRzIG9mIGVhY2ggY29sdW1uCmhlYWQobXBnKSAjIHZpc3VhbGlzZSBmaXJzdCA2IHJvd3Mgb2YgZGF0YQp0YWlsKG1wZywxMCkgIyB2aXN1YWxpc2UgbGFzdCAxMCAob3IgTikgcm93cyBvZiBkYXRhIApuYW1lcyhtcGcpICMgZ2V0IGNvbHVtbiBuYW1lcwpjbGFzcyhtcGcpICMgY2xhc3Mgb2YgZGF0YSBmcmFtZQpjbGFzcyhtcGckbWFudWZhY3R1cmVyKSAjIGNsYXNzIG9mIGRhdGEgY29sdW1uCm1wZyRkaXNwbCAjIHByaW50IGEgY29sdW1uCm1wZyRod3kgIyBwcmludCBhIGNvbHVtbgpgYGAKXCAgICAKCiMgQ3JlYXRpbmcgYSBwbG90IHdpdGggYGdncGxvdGAgIApbU2VjdGlvbiAzLjIuMl0oaHR0cHM6Ly9yNGRzLmhhZC5jby5uei9kYXRhLXZpc3VhbGlzYXRpb24uaHRtbCNjcmVhdGluZy1hLWdncGxvdCkgICAgIAoKKiBgZ2dwbG90KClgIENyZWF0ZXMgYSBjb29yZGluYXRlIHN5c3RlbSBmb3IgdXMtLWJhc2ljYWxseSBhbiBlbXB0eSBncmFwaC4gIAoqIGBnZW9tX3BvaW50KClgIEFkZHMgYSAibGF5ZXIiLCBlLmcuIGdlb21fcG9pbnQgKGJ1dCB0aGVyZSBhcmUgbWFueSBmb3IgZGlmZmVyZW50IGtpbmRzIG9mIGdyYXBocykuICAKClBsb3QgdHdvIG9mIHRoZSBkYXRhIGNvbHVtbnMgIApgYGB7ciwgZXZhbCA9IFR9CmdncGxvdChkYXRhID0gbXBnKSArIAogIGdlb21fcG9pbnQobWFwcGluZyA9IGFlcyh4ID0gZGlzcGwsIHkgPSBod3kpKQpgYGAKXCAgCgpDaGFuZ2luZyB0aGUgZGF0YSBjb2x1bW4gaW5wdXRzIGZvciB0aGUgKip4KiogYW5kICoqeSoqIGF4aXMgb2YgdGhlIHBsb3QgIApgYGB7ciwgZXZhbCA9IFR9CmdncGxvdChkYXRhID0gbXBnKSArIAogIGdlb21fcG9pbnQobWFwcGluZyA9IGFlcyh4ID0gY2xhc3MsIHkgPSBkcnYpKQpgYGAKXCAgCgpBc3NpZ24gZGF0YSB0byB2YXJpYWJsZXMgdG8gY3JlYXRlIGR5bmFtaWMgaW5wdXRzICAKYGBge3IsIGV2YWwgPSBUfQpteV9kYXRhIDwtIG1wZyAjIGNyZWF0ZSBvd24gdmFyaWFibGUgdXNpbmcgYSBuYW1lIG9mIHlvdXIgY2hvaWNlICAKCmdncGxvdChkYXRhID0gbXlfZGF0YSkgKyAKICBnZW9tX3BvaW50KG1hcHBpbmcgPSBhZXMoeCA9IGRpc3BsLCB5ID0gaHd5KSkKYGBgClwgIAoKIyMgVGhlbWVzICAKCkNoYW5nZSBwbG90IHN0eWxlLiBMaW5rIGZvciBtb3JlIFtnZ3Bsb3QgdGhlbWVzXShodHRwczovL3d3dy5kYXRhbm92aWEuY29tL2VuL2Jsb2cvZ2dwbG90LXRoZW1lcy1nYWxsZXJ5LykuICAgICAgICAKYGBge3IsIGV2YWwgPSBUfQpyZXF1aXJlKGdndGhlbWVzKQoKIyBjbGFzc2ljIHRoZW1lCmdncGxvdChkYXRhID0gbXlfZGF0YSkgKyAKICBnZW9tX3BvaW50KG1hcHBpbmcgPSBhZXMoeCA9IGRpc3BsLCB5ID0gaHd5KSkgKyAKICB0aGVtZV9taW5pbWFsKCkKCiMgbWluaW1hbCB0aGVtZQpnZ3Bsb3QoZGF0YSA9IG15X2RhdGEpICsgCiAgZ2VvbV9wb2ludChtYXBwaW5nID0gYWVzKHggPSBkaXNwbCwgeSA9IGh3eSkpICsgCiAgdGhlbWVfdHVmdGUoKSAgCgojIGFzc2lnbiB0aGVtZSB0byB2YXJpYWJsZSAgCm15X3RoZW1lIDwtIHRoZW1lX2NsYXNzaWMoKSAKZ2dwbG90KGRhdGEgPSBteV9kYXRhKSArIAogIGdlb21fcG9pbnQobWFwcGluZyA9IGFlcyh4ID0gZGlzcGwsIHkgPSBod3kpKSArIAogIG15X3RoZW1lICMgYXBwbHkgeW91ciBjaG9zZW4gdGhlbWUgIAoKYGBgClwgIAoKIyBBZXN0aGV0aWMgbWFwcGluZyAgCltTZWN0aW9uIDMuM10oaHR0cHM6Ly9yNGRzLmhhZC5jby5uei9kYXRhLXZpc3VhbGlzYXRpb24uaHRtbCNhZXN0aGV0aWMtbWFwcGluZ3MpICAKCmBjb2xvdXJgLiBDaGFuZ2UgdGhlIGNvbG91ciBvZiB0aGUgZGF0YSBwb2ludHMuCmBzaXplYC4gQ2hhbmdlIHRoZSBzaXplIG9mIHRoZSBkYXRhIHBvaW50cy4gIApgYWxwaGFgLiBDaGFuZ2UgdGhlIHRyYW5zcGFyZW5jeSBvZiB0aGUgZGF0YSBwb2ludHMuICAgCgojIyBDb2xvdXIKCkNvbG91ciBieSBjb2xvdXIgbmFtZS4gCmBgYHtyLCBldmFsID0gVH0KZ2dwbG90KGRhdGEgPSBteV9kYXRhKSArIAogIGdlb21fcG9pbnQobWFwcGluZyA9IGFlcyh4ID0gZGlzcGwsIHkgPSBod3kpLCBjb2xvdXIgPSAibGlnaHQgYmx1ZSIpICsKICBteV90aGVtZQogIApgYGAKXCAgIAoKQ29sb3VyIGJ5IGEgW2hleCBjb2RlXShodHRwczovL2h0bWxjb2xvcmNvZGVzLmNvbS9jb2xvci1waWNrZXIvKSBpbiBxdW90ZXMuICAKYGBge3IsIGV2YWwgPSBUfQpnZ3Bsb3QoZGF0YSA9IG15X2RhdGEpICsgCiAgZ2VvbV9wb2ludChtYXBwaW5nID0gYWVzKHggPSBkaXNwbCwgeSA9IGh3eSksIGNvbG91ciA9ICIjQkI1QzQyIikgKwogIG15X3RoZW1lCmBgYApcICAKCmNvbG91ciBieSBkYXRhIGNvbHVtbiAgCmBgYHtyLCBldmFsID0gVH0KZ2dwbG90KGRhdGEgPSBteV9kYXRhKSArIAogIGdlb21fcG9pbnQobWFwcGluZyA9IGFlcyh4ID0gZGlzcGwsIHkgPSBod3ksIGNvbG91ciA9IGNsYXNzKSkgKwogIG15X3RoZW1lCmBgYApcICAKCkluc2lkZSB2ZXJzdXMgb3V0c2lkZSB0aGUgYGFlc2AgIApgYGB7ciwgZXZhbCA9IFR9CmdncGxvdChkYXRhID0gbXlfZGF0YSkgKyAKICBnZW9tX3BvaW50KG1hcHBpbmcgPSBhZXMoeCA9IGRpc3BsLCB5ID0gaHd5LCBjb2xvdXIgPSAiYmx1ZSIpKSArCiAgbXlfdGhlbWUKYGBgClwgIAoKIyMgU2l6ZSAKClNpemUgYnkgaW50ZWdlciAgCmBgYHtyLCBldmFsID0gVH0KZ2dwbG90KGRhdGEgPSBteV9kYXRhKSArIAogIGdlb21fcG9pbnQobWFwcGluZyA9IGFlcyh4ID0gZGlzcGwsIHkgPSBod3ksIHNpemUgPSA1KSkgKwogIG15X3RoZW1lCmBgYApcICAKClNpemUgYnkgZGF0YSBjb2x1bW4gICAgICAKYGBge3IsIGV2YWwgPSBUfQpnZ3Bsb3QoZGF0YSA9IG15X2RhdGEpICsgCiAgZ2VvbV9wb2ludChtYXBwaW5nID0gYWVzKHggPSBkaXNwbCwgeSA9IGh3eSwgc2l6ZSA9IGNsYXNzKSkgKwogIG15X3RoZW1lCmBgYApcICAKCldlIGdldCBhIHdhcm5pbmcsIGJ1dCB0aGlzIGlzIG9rYXkuICAKXCAgCgojIyBUcmFuc3BhcmVuY3kgCmBgYHtyLCBldmFsID0gVH0KIyBtYXAgY2xhc3NlIGNvbHVtbiB0byBkaWZmZXJlbnQgdHJhbnNwYXJlbmNpZXMKZ2dwbG90KGRhdGEgPSBteV9kYXRhKSArIAogIGdlb21fcG9pbnQobWFwcGluZyA9IGFlcyh4ID0gZGlzcGwsIHkgPSBod3ksIGFscGhhID0gY2xhc3MpKSArIAogIG15X3RoZW1lCmBgYApcICAKCiMjIFNoYXBlICAKYGBge3IsIGV2YWwgPSBUfQpnZ3Bsb3QoZGF0YSA9IG15X2RhdGEpICsgCiAgZ2VvbV9wb2ludChtYXBwaW5nID0gYWVzKHggPSBkaXNwbCwgeSA9IGh3eSwgc2hhcGUgPSBjbGFzcykpICsKICBteV90aGVtZQpgYGAKXCAgCgpBbnkgd2FybmluZ3M/IFllcywgYmVjYXVzZSBzaGFwZSBtYXhlcyBvdXQgYXQgc2l4IGxldmVscy4KXCAgCgojIyBNYW51YWxseSBjaGFuZ2luZyBhZXN0aGV0aWMgcHJvcGVydGllcwogIApCdXQgd2UgY2FuICpzZXQqIHRoZSBhZXN0aGV0aWMgcHJvcGVydGllcyBtYW51YWxseSwgaW5zdGVhZCBvZiBoYXZpbmcgZ2dwbG90IGRvIHRoZSBzY2FsaW5nIGF1dG9tYXRpY2FsbHkuIEZvciBleGFtcGxlLCB3ZSBjYW4gbWFrZSBvdXIgZ2dwbG90IHBvaW50cyBhbGwgYmx1ZSBsaWtlIHRoaXMuIFRoaXMgdGltZSwgcHV0dGluZyBjb2xvdXIgT1VUU0lERSB0aGUgYGFlc2AgYXJndW1lbnQuCgpgYGB7ciwgZXZhbCA9IFR9CmdncGxvdChkYXRhID0gbXlfZGF0YSkgKyAgCiAgZ2VvbV9wb2ludChtYXBwaW5nID0gYWVzKHggPSBkaXNwbCwgeSA9IGh3eSksIGNvbG91ciA9ICJibHVlIikgKwogIG15X3RoZW1lCmBgYApcICAKClVzaW5nIGNvbG91ciBib3RoIGluc2lkZSBhbmQgb3V0c2lkZSB0aGUgYWVzICAKYGBge3IsIGV2YWwgPSBUfQpnZ3Bsb3QoZGF0YSA9IG15X2RhdGEpICsgCiAgZ2VvbV9wb2ludChtYXBwaW5nID0gYWVzKHggPSBkaXNwbCwgeSA9IGh3eSwgY29sb3VyID0gY2xhc3MpLCBjb2xvdXIgPSAiI0FFNDJCQiIpICsKICBteV90aGVtZQpgYGAKXCAgCgoqKlRoZSBpbm5lciBvbmUgaXMgb3ZlcnJpZGRlbi4qKiAgICAgClwgIAoKUHV0dGluZyBpdCBhbGwgdG9nZXRoZXIgYXMgYSBzbmFwc2hvdCBvZiB3aGF0J3MgcG9zc2libGUgIApgYGB7ciwgZXZhbCA9IFR9CmdncGxvdChkYXRhID0gbXlfZGF0YSkgKyAKICBnZW9tX3BvaW50KG1hcHBpbmcgPSBhZXMoeCA9IGRpc3BsLCB5ID0gaHd5LCBjb2xvdXIgPSBjbGFzcywgc2l6ZSA9IGNsYXNzLCBhbHBoYSA9IGNsYXNzKSkgKwogIG15X3RoZW1lCmBgYAoKIyMgQWVzdGhldGljcyB5b3UgY2FuIG1hbnVhbGx5IHNldCAgCgogICogVGhlIG5hbWUgb2YgYSBjb2xvdXIgYXMgYSBjaGFyYWN0ZXIgc3RyaW5nLgogICogVGhlIHNpemUgb2YgYSBwb2ludCBpbiBtbS4KICAqIFRoZSBzaGFwZSBvZiBhIHBvaW50IGFzIGEgbnVtYmVyLCBhcyBzaG93biBpbiBGaWd1cmUgMy4xLiAKICAKICA8IS0tIC0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tIGltYWdlIC0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLSAtLT4KPGRpdiBhbGlnbj0iY2VudGVyIj4KICA8aW1nIHNyYz0iaW1nL3NoYXBlcy5wbmciIHN0eWxlPXdpZHRoOjEwMCU+CjwvZGl2Pgo8IS0tIC0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tIGltYWdlIC0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLSAtLT4KXCAgCiAgCmBSYCBoYXMgMjUgYnVpbHQgaW4gc2hhcGVzIHRoYXQgYXJlIGlkZW50aWZpZWQgYnkgbnVtYmVycy4gVGhlcmUgYXJlIHNvbWUgc2VlbWluZyBkdXBsaWNhdGVzOiBmb3IgZXhhbXBsZSwgMCwgMTUsIGFuZCAyMiBhcmUgYWxsIHNxdWFyZXMuIFRoZSBkaWZmZXJlbmNlIGNvbWVzIGZyb20gdGhlIGludGVyYWN0aW9uIG9mIHRoZSBgY29sb3VyYCBhbmQgYGZpbGxgIGFlc3RoZXRpY3MuIFRoZSBob2xsb3cgc2hhcGVzICgwLS0xNCkgaGF2ZSBhIGJvcmRlciBkZXRlcm1pbmVkIGJ5IGBjb2xvdXJgOyB0aGUgc29saWQgc2hhcGVzICgxNS0tMTgpIGFyZSBmaWxsZWQgd2l0aCBgY29sb3VyYDsgdGhlIGZpbGxlZCBzaGFwZXMgKDIxLS0yNCkgaGF2ZSBhIGJvcmRlciBvZiBgY29sb3VyYCBhbmQgYXJlIGZpbGxlZCB3aXRoIGBmaWxsYC4gIAoKXG5ld3BhZ2UgICAgICAKCiMgRnVydGhlciBwbG90dGluZyBleGFtcGxlcyAgICAKW1NlY3Rpb24gMy4zLjFdKGh0dHBzOi8vcjRkcy5oYWQuY28ubnovZGF0YS12aXN1YWxpc2F0aW9uLmh0bWwjZXhlcmNpc2VzLTEpICAKClRoZSBvbmxpbmUgcmVmZXJlbmNlIGNvbnRhaW5zIGZ1cnRoZXIgZXhhbXBsZXMgb2YgaG93IHRvIHZpc3VhbGlzZSB5b3VyIGRhdGEuICAgIAoKIyBSZWFkaW5nIGluIG91dHNpZGUgZGF0YTogTllDIEFpcmJuYiBkYXRhICAKCmBgYHtyLCBldmFsID0gVCwgZWNobyA9IEZ9CmxpYnJhcnkodGlkeXZlcnNlKSAjIGluY2x1ZGVzIHBhY2thZ2UgInJlYWRyIgojIEFsbCBBaXJibmIgZGF0YSAoMTA2IGNvbHMpCnVybCA8LSAiaHR0cDovL2RhdGEuaW5zaWRlYWlyYm5iLmNvbS91bml0ZWQtc3RhdGVzL255L25ldy15b3JrLWNpdHkvMjAyMS0wNC0wNy9kYXRhL2xpc3RpbmdzLmNzdi5neiIKCm55Y19mdWxsIDwtIHJlYWRfY3N2KHVybCkgIyByZWFkcyBpbiBkYXRhCmhlYWQobnljX2Z1bGwpCmBgYApcICAKClVzaW5nIGEgc21hbGxlciBkYXRhc2V0IApgYGB7ciwgZXZhbCA9IFQsIHJlc3VsdHM9ImhpZGUifQojIHNtYWxsZXIgY3N2IGZpbGUgKDE2IGNvbHMpCnVybCA8LSAiaHR0cDovL2RhdGEuaW5zaWRlYWlyYm5iLmNvbS91bml0ZWQtc3RhdGVzL255L25ldy15b3JrLWNpdHkvMjAyMS0wNC0wNy9kYXRhL2xpc3RpbmdzLmNzdi5neiIKCm55YyA8LSAgcmVhZF9jc3YodXJsKQpueWMgPC0gbnljW255YyRpZCA8IDIwMDAwLF0gIyBnZXQgc21hbGxlciBzdWJldCBvZiBkYXRhCmxlbmd0aChueWMkaWQpICMgcHJpbnQgbGVuZ3RoIG9mICdpZCcgY29sdW1uCmhlYWQobnljKQpgYGAgClwgICAgCgojIFBsb3R0aW5nIEFpckJuQiBkYXRhIHdpdGggZ2dwbG90ICAKClVzaW5nIHRoZSBhYm92ZSBwbG90dGluZyBmdW5jdGlvbnMgdG8gdmlzdWFsaXNlIHRoZSBBaXJCbkIgZGF0YSAgCgpgYGB7ciwgZXZhbCA9IFR9CiMgcGxvdCBuZWlnaGJvcmhvb2RfZ3JvdXAgdnMgcHJpY2UKZ2dwbG90KGRhdGEgPSBueWMpICsgCiAgZ2VvbV9wb2ludChtYXBwaW5nID0gYWVzKHggPSBuZWlnaGJvdXJob29kX2dyb3VwX2NsZWFuc2VkLCB5ID0gcHJpY2UsIGNvbG91ciA9CiAgICAgICAgICAgICAgICAgICAgICAgICAgICAgbmVpZ2hib3VyaG9vZF9ncm91cF9jbGVhbnNlZCksIHNoYXBlID0gMjEsIHN0cm9rZSA9IDEpICsKICBteV90aGVtZQpgYGAKCmBgYHtyLCBldmFsPVR9CiMgcGxvdCBtaW5pbXVtX25pZ2h0cyB2cyBwcmljZQpnZ3Bsb3QoZGF0YSA9IG55YykgKyAKICBnZW9tX3BvaW50KG1hcHBpbmcgPSBhZXMoeCA9IG1pbmltdW1fbmlnaHRzLCB5ID0gcHJpY2UsIAogICAgICAgICAgICAgICAgICAgICAgICAgICBjb2xvdXIgPSBuZWlnaGJvdXJob29kX2dyb3VwX2NsZWFuc2VkKSwgc2hhcGUgPSAyMCwgc2l6ZSA9IDMsIHN0cm9rZSA9IDEpICsKICBteV90aGVtZQpgYGAKCmBgYHtyLCBldmFsPVR9CiMgYXZhaWxhYmlsaXR5XzM2NSB2cyBwcmljZQpnZ3Bsb3QoZGF0YSA9IG55YykgKyAKICBnZW9tX3BvaW50KG1hcHBpbmcgPSBhZXMoeCA9IGF2YWlsYWJpbGl0eV8zNjUsIHkgPSBwcmljZSwgCiAgICAgICAgICAgICAgICAgICAgICAgICAgIGNvbG91ciA9IG5laWdoYm91cmhvb2RfZ3JvdXBfY2xlYW5zZWQpLCBzaGFwZSA9IDIxLCBzdHJva2UgPSAxKSArCiAgbXlfdGhlbWUKYGBgCiAgICAKYGBge3IsIGV2YWw9VH0KIyBwbG90IGxvbmdpdHVkZSB2cyBwcmljZQpnZ3Bsb3QoZGF0YSA9IG55YykgKyAKICBnZW9tX3BvaW50KG1hcHBpbmcgPSBhZXMoeCA9IGxvbmdpdHVkZSwgeSA9IHByaWNlLCAKICAgICAgICAgICAgICAgICAgICAgICAgICAgY29sb3VyID0gbmVpZ2hib3VyaG9vZF9ncm91cF9jbGVhbnNlZCksIHNoYXBlID0gMjEsIHN0cm9rZSA9IDEpICsKICBteV90aGVtZQpgYGAKCgpUcnkgeW91ciBvd24gcGxvdCB1c2luZyB0aGUgb3RoZXIgdmFyaWFibGVzIGluIHRoZSBkYXRhc2V0ICAgIApgYGB7cn0KIyBwbG90IG5laWdoYm9yaG9vZF9ncm91cCB2cyBwcmljZQpuYW1lcyhhaXJibmIpCmdsaW1wc2UoYWlyYm5iKQoKbXlfZGF0YSA8LSBOVUxMCnggPC0gTlVMTAp5IDwtIE5VTEwKY29sb3VyIDwtIE5VTEwKc2hhcGUgPC0gTlVMTApzdHJva2UgPC0gTlVMTApgYGAKCmBgYHtyLCBlY2hvPUZ9CmdncGxvdChkYXRhID0gbXlfZGF0YSkgKyAKICBnZW9tX3BvaW50KG1hcHBpbmcgPSBhZXMoeCA9IHgsIHkgPSB5LCBjb2xvdXIgPSBjb2xvdXIpLCBzaGFwZSA9IHNoYXBlLCBzdHJva2UgPSBzdHJva2UpCmBgYAoKPCEtLSBlbmQgYm9keSAtLT4KPCEtLSBfX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fIC0tPgo8IS0tIF9fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX18gLS0+CjwhLS0gX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fXyAtLT4KCiAgIAo=