For generating the visualizations we used d3.js in a Vue.js web application with privacy-respecting client-side rendering. For designing the visualizations we created a custom-made sample data generator to help us design for edge cases.
Most of those graphics are using SVGs (Scalable Vector Graphics). From a brief glimpse at the code, it looks like they are being controlled by a JavaScript library called D3, which aids in visualizing datasets.
I agree with you that CSS is kinda poopy though.
Also this website is really well done, as others have said.
Dank! Goede vragen en suggesties.
Wil er niet echt iets mee aantonen, is vooral een oefening om D3 te gebruiken. Lijnen laten staan heb ik geprobeerd maar dat werkt niet echt want na een tijdje zie je er niks meer bijkomen. Maar dikker laten worden is wel een goed idee ja. Kan ze misschien ook langzaam laten vervagen, dan blijven de veel gebruikte routes staan.
Looks like the D3.js library is being used so it shouldn't be too hard to change the color of a single node by modifying the source located in http://flowingdata.com/projects/2015/timeuse-simulation/index.html.
Edit: So it looks like an easy way to make one dot stand out is to change the radius in javascript to:
// A node for each person's schedule var nodes = sched_objs.map(function(o,i) { var act = o[0].act; act_counts[act] += 1; var init_x = foci[act].x + Math.random(); var init_y = foci[act].y + Math.random(); return { act: act, radius: i == 0 ? 32 : 3, x: init_x, y: init_y, color: color(act), moves: 0, next_move_time: o[0].duration, sched: o, } });
The variable i represents the index of the node in the list of nodes. The radius used to always be set to 3, but I modified it so that the first node gets a radius of 32. This is pretty large so it gets pushed around by the other nodes. If you'd like a random node to change size, you can modify the check i == 0 to instead compare the index to a random number within bounds. I leave that as an exercise to the reader.
Also, you could maybe add another field to be something like isSpecial and then check for the presence of that field in the color() function. I think you can also add outlines to a node easily in d3 so that may be a better approach.Or you can modify the data to limit the number of nodes entirely.
Bonus: modify i == 0 to i % 25 == 0
and lowering the radius would probably be a good idea too
Managed to find something called D3JS, maybe that's what this is referring to?
> D3.js is a JavaScript library for manipulating documents based on data. D3 helps you bring data to life using HTML, SVG, and CSS. D3’s emphasis on web standards gives you the full capabilities of modern browsers without tying yourself to a proprietary framework, combining powerful visualization components and a data-driven approach to DOM manipulation.
I used d3. My messy chart code probably isn't the best place to start if you want to learn; this is a good intro book.
You'll want to look into D3: https://d3js.org/
For example: https://bl.ocks.org/mbostock/4063423
There's a D3 pocket primer in the current humble bundle: https://www.humblebundle.com/books/pocket-primers-books
Honestly I'd rather stab my eyes out with icepicks than write GUI software in Python and I say that as a 10 year python dev. It doesn't have a good impedance match with ANY toolkit and NONE of the available toolkits support 2 way data binding AFAIK.
I'd write the GUI layer in the toolkits native language (eg. c++ for QT) and embed the python interpreter or call with RPC.
Or just write a web app and use D3.js for visualization. D3 uses a declarative language to bind data to DOMs. It can render HTML, SVG, and canvas.
Here's a POC web app I wrote that used a python backend to stream live data via websockets to a D3 based browser frontend https://github.com/n1ywb/wavefront-web/blob/master/wavefrontweb/static/js/main.js
De broncode kan je hier vinden: https://gitlab.com/guushoekman/train-journeys
Zeer korte uitleg:
You can try to compile the Python code to webassembly. But honestly, I don't think manim is a good choice here.
You want:
Manim can do the following:
For all the other points you would have to fight really hard or make compromises.
I think you are better of using d3js or something that suits you. If you really want to copy the 3b1b style (which is great, but not the only visual style out there that looks good) you can most certainly achieve something similar with most libraries out there.
d3, one of the main data visualization tools available, was developed in large part by Mike Bostock, initially while at NYTimes. They got very good at it.
https://www.chartjs.org is free and relatively nice out of the box but the clean and modern piece isn't that great.
https://d3js.org is also free but not as nice out of the box. If you can get a hang of it you can make awesome stuff. A lot of the best data visualizations you see on NYT and other popular media are d3. You can add endless interactions, etc. you just need to figure out how to do it which is the tricky part.
He's wrong in basically saying that you have to be a full stack developer.
What he's right about is that being able to visualize your work is valuable and something worth investing time in.
Lots data science programs will require at least one course on visualization. Something like D3 isn't uncommon and doesn't require a full stack of web dev experience.
This is d3.js. Quite possibly done by the creator of d3, as he at least used to work for NYT. It is also possible that the data manipulation was done in R. There are a few ways of presenting R work in d3.js. Most commonly through Plotly. Although that will never compare to working with d3 natively.
The blue region represents daytime. The white lines represent when the moon is above the horizon (the time between moonrise and moonset). The opacity of the moon lines represents the fraction of the visible moon that is illuminated. New and full moons are labeled on the right axis.
Source: Vladimir Agafonkin’s suncalc.
Tool: D3.js
D3, the main javaScript library I'm using, is created for data visualization and has some build-in functions to animate data. I did not use this possibility a lot, but with this population bars animation looks really good. And thanks for d3 creators, it's just a few lines of code. So it will not take a lot of time to add animation to some other layers, but I'm not sure if benefits are sufficient when there is a bunch of 'core' functions now working properly. I will wait for your feedback and decide whether I should add more animations or it can wait for some months.
D3.js would be a better option, it is designed for visualizing data. I concur with /u/LankyCyril, if you'd be willing to post the data that you worked hard to collect, I'm sure someone might take the time to get something going, perhaps even on github so anyone can fork and work off of it.
D3 takes a bit of focused practice to get comfortable with, but opens a lot of possibilities if you are even slightly interested in data visualisation. Think of it as the Swiss army knife of translating data to the DOM.
The problem with the current pricing model is the page is going to be mostly blank for a long time.
Just put the GO BCH! on the view and say half the page is now worth $1 (0.0018 BCH), 1/4 is 0.009, etc. Then the "real estate" RATE is always the TOTAL FEES / 1M. Each graphic can then be scaled to fit within their respective size: FEE * RATE = AREA. Use the AREA x picture's aspect ratio (like 4/3) and you get the allotted image size.
Bonus Points: Read up on SVG graphics in HTML. It can do all graphic work of scaling and placing the image assets for you. You would simply add another graphic to the mix with the respective fee, and the browser would do the rest. Each graphic could also have a URL, making the images clickable links.
This way, participants could send you a source picture of any size or dimensions and it would be automatically scaled and positioned on the page by the browser. Throw in a little Responsive Design magic and you would get best fit for the browser window size.
Elite Version: Use a presentation from d3js so that the images self organize, like d3/temporal-force-directed-graph
I see... That's actually a fairly ambitious/complex project if you only have some basic PHP knowledge, but it can certainly be done.
If you want to be able to manually enter data about individual currencies, you might consider a currency custom post type. The list page would just be the standard post type archive page and the chart page would be the singular page template.
If it will be entirely API driven (which would probably make more sense given the number of available currencies), then you will want to read up on the rewrite API to add custom endpoints to your site.
For the live charts: You may be able to find a site that offers embeds (I don't see anything obvious on cryptocompare).
If not you now to add some solid javascript knowledge to the requirements. You can probably utilize one of many available charting libraries to simplify the project a bit (e.g. d3, not sure if it is the most appropriate, you may be able to find a library specifically tailored to stock/currency charts).
From there you would either use the histo*
API endpoints or the websocket API to check for new data and update the charts accordingly.
Alternatively, consider the fact that WordPress doesn't bring much to the table for this type of project and therefore may not be the best fit.
That sounds great. I hope you enjoy teaching the course, even under the circumstances. The material is enjoyable for students, and a nice transition between computational courses and proof-based courses.
Let me know how the interactive topograph goes. Steven Sam created something similar in Java about 10 years ago but I don't know if the code is still around. I was going to try to learn d3.js and make some interactive visualizations, but that will probably wait until 2018, and I don't know if I'll get the topograph-walking to work.
There might be some interesting challenges implementing a navigable Poincare disk. Maybe you've thought about them already. Basically, I think one should store points in the Poincare disk in a custom format, since so much takes place near the boundary. Otherwise, you might fall victim to some bad rounding errors.
If you use the inspect feature in Chrome, it looks to be built on D3.js.
Completely agree with your point on what the map is. It's meant to show geographic variation. Not to show absolutes in the overall voting.
Discrete mathematics is pretty foundational to computing -- which is unfortunate because high schools traditionally (in the US at least) go for the calculus route instead.
Of the topics in that article, logic, set theory, combinatorics, and graph theory are probably the most relevant because they help model the relationships of real world things. Relationships between things is way more common of a problem than finding areas under curves.
The books I'd recommend are all textbooks which aren't great for self learning, but you could try your local library to see if any call out to you on those topics so you're more motivated to read them. You could try looking for stuff paired with D3 or Processing which are often used to build art installations.
It seems that your comment contains 1 or more links that are hard to tap for mobile users. I will extend those so they're easier for our sausage fingers to click!
Here is link number 1 - Previous text "jit"
Here is link number 2 - Previous text "d3"
^Please ^PM ^/u/eganwall ^with ^issues ^or ^feedback! ^| ^Delete
Maybe it's the kind of thing I should've put in its own post, but I created something very similar a bit ago also by scraping data from j-archive, but using d3js to create the heatmap (and a few other bits of info). It has a slider that allows you to select a specific season, or a button to show data for all the seasons together.
Alluvial or Sankey diagram (there are small differences but their often called each other), several methods exist depending on platform and previous knowledge, R can craft one with the right plugins, theirs an add-on for google sheets, RawGraphs, theres a plugin for javascript. to name a few
> Recently I have begun experimenting with Shiny, Bokeh, and Dash (which are interactive frameworks in R and Python for creating dashboards and displaying data interactively), and I have found most of these packages are actually just wrappers for Javascript libraries.
Actually Shiny and Bokeh are wrappers for D3.js. I don't know anything about Dash, though.
D3 is the gold standard for data visualisation in JavaScript. There is a really good book, Data Visualisation with Python and JavaScript, by Kyran Dale on O'Reilly. It's designed for devs who know a little of either Python or JavaScript, and the first few chapters go over the basics of both, comparing one to the other. Then it goes through a Python web scraping, data cleaning, web back end technology stack, then turns the data you scraped into something beautiful and interactive with D3.
Or if you'd rather just deep dive into D3, I can recommend Interactive Data Visualisation for the Web by Scott Murray, also on O'Reilly. Despite the title, the book doesn't spend long discussing web development, just the minimum HTML you need to make your D3 projects display. The book is an expanded tutorial on D3, from the very basics to some of the really far-out features.
Either one of these books will give you exactly what you're looking for, depending how much time you'd want to spend with Python before getting into visualisation. And by the way, D3 can also do maps.
There are several charting libraries for JavaScript that would do most of the work for you. The grandaddy of them all is D3, which can do anything, but with a very steep learning curve. If you just want quick results, try a library like C3 or Highcharts, which use D3 behind the scenes, but only give you a few options to work with.
The data for this map comes from a study published earlier this month in Scientific Data, which compiled data on historical city populations going back as far as 3700 BC.
The data is not comprehensive: certain parts of world are better represented than others, and some well known cities do not appear until long after they were founded. That said, the authors compiled a pretty impressive amount of data over a long time period.
The map was built using D3 with Canvas and SVG.
d3js probably - d3js.org
One of their example visualizations -
https://observablehq.com/@johnburnmurdoch/bar-chart-race-the-most-populous-cities-in-the-world
If you need to convert all your R code to JS, then you will have to do it by hand.
If it's fine to have an R backend powering the charts, then this is certainly doable - have a Shiny server serve up charts or snippets that the frontend can use. This would be my preferred way.
You can also take a look at the htmlwidgets package, which allows you to generate HTML + JS code from R. There are bindings for JS vizualization libraries such as D3.js already - see the "reverse imports" section.
Alternatively, you can have R output the data as eg. JSON, and then the frontend guys can choose a JS charting framework of their liking to make the charts, calling your API for the data.
If you have experience in JS it might be best to start looking at the most popular charting / visualization frameworks. The two that immediately jump to mind are:
If I was in your situation I'd look at the two of these, get a rough idea of what they can do and then create lots of wireframes. You should be able to get some funding off your wireframes and a business plan, enough to get a POC created then target getting some real funding.
I copied it all from this table https://overwatchleague.com/en-us/players so it wasn't too much work to gather.
The map and points were drawn mostly with D3 (a Javascript library, not Diablo 3). Rest of the page done mostly in the VueJS framework.
We used pixijs for creating the animation and d3 for the charts. You can find the data in this spreadsheet
One of the big problems that Martin O'Leary talked about in his map generation posting was the difficulty of placing labels. One solution to that problem is called "force layout". The basic idea is that you treat the labels and other map elements as objects with forces acting on them, and let them move around until they find a "comfortable" spot. For example, you might make labels move away from rivers (so they don't cover them up). It turns out that D3js (which Martin's map generator uses) has a force layout engine built-in. Unfortunately, in the latest version it doesn't have the kind of collision force most useful for laying out labels. So I had to write that new force, but the result works pretty well. The attached album shows the layout process in more detail. (I think I made the images a bit too large, so you might have to shrink the size in your browser for best viewing.)
This website doesn't seem to use it, but I think it's also possible using D3.js: https://d3js.org
The first thing you have to do is increase the contrast by a lot using Photoshop, so it's mostly black and white. Then use Javascript to convert the jpg/png to an array of colors (http://stackoverflow.com/questions/8751020/how-to-get-a-pixels-x-y-coordinate-color-from-an-image). Now you need to create a loop to check each pixel whether its color is closer to black and change it to #000000, or closer to white and change it to #ffffff. At this moment you will have a matrix of your original image.
Then you can take a look at the follow example, to make the blobs react to the mouse pointer and to add a collision detection: http://mbostock.github.io/d3/talk/20111018/collision.html
I haven't done it before, but this is how I would try it, so I hope this can get you started.
Uma forma de representar isso é um mapa interativo com D3.js. Quando a pessoa bota o mouse em cima de um estado, ele mostra somente as linhas indo praquele estado (mas não as linhas saindo dele). Se tirar o mouse de cima mostra tudo como você fez.
É mais complicado mas não to conseguindo imaginar uma forma estática de representar isso.
E ah, nem isso é suficiente porque tem linhas dentre o mesmo estado.
If you want to make interesting visualizations I would recommend checking out Chrome Experiments where a lot of the demos have source available to play with. There are also a few audio + visual demos in there.
Some other really fun visualization tools that you can get started pretty quick with are A-Frame and d3.js.
I'm currently in the Data Analyst nano degree on Udacity, and this visualization is my project for the "Data Visualization in d3.js" class.
I scraped the data from congress.gov because the only datasets I could find elsewhere had flaws with them. The main tool I used is d3.js. All of the code and my writeup about the entire process can be found here.
URL: www.stateliquorlaws.com
Purpose: Track the wide range of alcohol laws across the United States
Technologies Used: Back end: Flask (Python), Front end: d3.js, CSS/Bootstrap, Google Maps API
Feedback Requested: Usability, general feedback on design, accuracy of content
Comments: This has been a pet project of mine for the past several months and I feel like it's finally come together. I most recently added a geolocation service (sorry, US only). To dos: many county-level state laws remain to be added.
I'm still using jQuery on 80% of my projects, and the reason for that isn't jQuery itself, it's DataTables
If you're developing any sort of business application you'll need to display data in a tabular fashion and I'm yet to find any javascript table library that is better than DataTables. It just works, fast to code against, and offers a comprehensive set of features that pretty much covers any business requirement. And it's fully mature and isn't going anywhere. Open to any suggestions for alternatives of course - just not found any yet.
DataTables is also the reason why I've not heavily adopted ReactNative - which is the first framework I have seriously considered making my standard toolkit (everything previous always looked too much like framework of the week)
It's not that I don't agree that jQuery is on the way out - I no longer automatically use it where I'm not using DataTables - I have projects based around libraries like MapBox and D3js where it doesn't get automatically included - but DataTables and several other goto widgets that I lean on all the time have jQuery dependencies and that seals the deal.
D3, "data driven documents" is a library for data visualisation in JavaScript. It drives, probably, most of the interesting data visualisations you see on the internet. From simple graphs and charts, to Sankey diagrams, force networks, geospatial stuff, heatmaps, treeviews, calendars, and probably whatever funky thing you can think of.
Click a hexagon and enjoy some data visualisation.
Or here: https://observablehq.com/@d3/gallery
Yes, WebGL would be overkill for this, unless you're planning to draw thousands of circles.
You have a few easier options:
- SVG: using circle, line and path primitives
- Canvas: using moveTo, lineTo, arc, etc.
Other options is using something like d3 to abstract your data and layout from your representation.
If you want to get started quick, I recommend canvas.
Hey,
i've had the same problem. Adding the type="text/javasript" where you load d3 fixed it for me:
<script type="text/javasript" src="https://d3js.org/d3.v4.min.js"></script>
Hope this helps.
So you're looking at data visualization. This is often done with Python, it has a lot of support and libraries dedicated to working with data, but can Javascript can also be used!
I'm not sure how much front/back-end you'd actually need to learn, depends on the work you'd be doing I suppose, but a few things you might want to focus on could be:
If you want to go heavier into the data-side, you'll probably want to focus more on D3 or another library. If you want to focus on displaying and creating dashboards instead then it'd be a mix of D3 and a front-end framework.
Either way, you'll want to learn the JS fundamentals first. In any case, building stuff like recommended by the other commenter is the way to go, you have lots of Udemy courses that can teach you the JS/D3 but make sure you get some practice in as well.
It's rainy season in California again so time to check on the status of California's reservoirs.
Data and Tools The data on water storage comes from the California Department of Water Resources' (DWR) Data Exchange Center. Python is used to extract the data from this page daily and wrangle the data in to a clean format. Visualization was done in javascript and specifically the D3.js visualization library. It was my first time using D3 and it took me a long time to get up to speed. It takes a fair amount of work to make graphs compared to other more plug-and-play libraries but its very customizable, which is a plus. It was the only tool that I could find that would allow me to make a vertical marimekko graph.
> IMO GUI's are great if you want to visualize something. We are human after all and we are biologically far better distinguishing visual patters rather than text.
Good GUIs use visualization well. There aren't that many good GUIs, though.
One of the first notable ones I can remember used WWW imagemaps. Even in the early years, the WWW was already far more powerful than most GUI toolkits. And it was quickly adopted for building UIs. Partly because it was fast to built web UIs, and just as much because it was flexible and portable and the server could trivially live on a different host than the UI. But also because it was more powerful than other GUIs. More powerful than Hypercard, which was already far better than a standard Mac GUI toolkit.
These days we use some web GUIs that leverage D3.js and vector SVG. No CLI can match what those can do. But very, very few classic GUIs do, either.
You could do it on the back end. You would have to generate an image of the chart, and then serve that image. It would be impossible to make it interactive, since the front-end would have none of the data, and the back end can't generate and serve it fast enough to make it seem interactive. But there's nothing stopping you from doing it.
You could even try to make it interactive by having <area/>
tags mapped to the image.
That would be about 1000 times harder than serving the data to a front end that had a simple JS plotting library. You can go high level library for the JS, something like charts, or you can jump straight into the lower level of D3. In either case, a little javascript never killed anyone. Intentionally. Ok that's probably a lie too, but you're not gonna get away from it! :P
​
You'll want d3.js for that. The library has a steep learning curve and can be intimidating at times, but that's because it includes lots of extra tools to try to give you everything you'd need in one package.
I'm a software developer and often create visualizations for the web applications I create. I'm not sure how much you know about development, but web apps are written in JavaScript and the main visualization library out there is D3.js. You can check out the examples page on that website to see the kinds of visualizations typically created in D3. If you were to go this route, you'd need to bunker down and learn JavaScript and the D3.js library.
I had a similar problem and bought this and a few other of his books The Visual Display of Quantitative Information https://www.amazon.com/dp/0961392142/ref=cm_sw_r_cp_apa_YjhABbA171C24
It's amazing what problems can be solved using good design. But it may exasperate some of your developers to recreate the methoods. But a lot of cool templates are available from here https://d3js.org
I'm not entirely sure about this, but after digging through their source code, I think they're using PixiJS along with a bunch of related extensions and other libraries (e.g. earcut...)
Other resources on the internet claim NYT as a whole uses react for their frontend, but I'm not too sure about that here -- I think in this specific case they maybe might be using backbone.js (along with a bunch of other libraries -- jQuery, Underscore, Hammer.js, Sizzle.js, Sock.js...). But the animations are all probably PixiJS.
I also know the NYT also sometimes uses D3.js, though I don't think they're using that particular library here.
What kind of application is it? How is the data stored?
There's not really frameworks for datavis. If you're building some sort of CRUD app you can use whatever backend you want and for a front end you may consider looking into d3j https://d3js.org
They are snippets of code written in javascript that use the visualization library d3.js: https://d3js.org ; to use them, you'd have to embed the javascript into an html page that you create, and load in your own data using the d3 mechanisms.
There is no interface to do this with your data so coding is required. The visualizations that are included in whichever dashboard software you're using are picked by the developers of that software; the Google people picked the visualizations they want to support (like the date heatmap) and load data from their tool into them for you.
probably just dump the data into a database rather than csv file and not have to worry about uploading it to the site. the site would just read from the database.
hardest part would be visualizing the data, but there are plenty of examples out there for doing crazy data visualization using d3js (https://d3js.org/).
either java or python is fine for the server-side code. you would need some type of database and need to learn about that.
you would need some amount of html, css and javascript knowledge for the actual site content.
"difficulty" for a web dev with a reasonable amount of experience? not very; it would probably take a week or two.
but considering you seem to know very little about any of this stuff ... anywhere from a couple months to a couple of decades.
E cam dificil cu super basic stuff. Dar daca iti doresti, click dreapta -> inspect (in chrome), si ai acolo script-ul folosit. Daca il parcurgi vezi ca datele sunt incarcate de aici: http://flowingdata.com/projects/2015/timeuse-simulation/data/days-simulated-v2.tsv Din modul cum sunt prelucrate trebuie sa-ti dai seama ce inseamna si apoi sa le modifici corespunzator. Daca vrei sa modifici vizualizarea sau sa intelegi mai bine cum e creata incepi aici: https://d3js.org/
That site is using d3.js for these visualizations. D3 is incredibly flexible, but with that comes a level of complexity much higher than the average charting library.
So aside from the standard web toolkit you need to learn (HTML/CSS/JS) you need to learn to use D3.js. And that's just to build the visualization.
If you want to learn how to crunch the data on the server, I recommend looking into learning data science.
Brother, you can get free as well as paid awesome graphs and charts from
https://d3js.org https://threejs.org
You can use any from these richfull library, they have huge no. of charts and other 3D web based objects.
I hope this might help you to get your work done 😁👍
A lot of the fancy, interactive ones are done using d3.js. There's quite a lot to take in, even if you're already familiar with web programming, but it will do pretty much anything you throw at it.
Thinking aloud: What about reading some existing work similar to what you want to write? I think D3.js' introduction is really good as an intro to a library, and <em>Nature of Code</em> to a technical concept, for example.
Hey! I actually majored in Philosophy (Wittgenstein is my homie)!
I've got about 3 years of javascript career experience under my belt, tho this is all web-development centered... so I'm not able to give you any suggestions outside of web-development or native applications. Honestly, It's going to be a bit of a struggle finding data-viz javascript tutorials that are totally outside of those worlds tho.
Anyways... If you arn't against web dev, by far the most popular JS data visualisation library is D3 ( https://d3js.org/ ). It has a steep learning curve, but once you've mastered it you're really able to do quite a lot.
As far as tutorials, I'd suggest this ( https://www.amazon.com/gp/product/1491921285/ref=as_li_qf_sp_asin_il_tl ). But whatever tutorial you end up following for D3, just be sure it's for version 4.X.
Hey guys!
The actual streamgraph itself was coded in a Javascript library called D3.js. (There's actually a tonne of mathematical stuff that needs to happen in order to draw those smooth lines!)
Originally I thought I could do this in just Excel, but oh how mistaken I was. The guys over at /r/visualization pointed me in the direction of D3.js - (https://www.reddit.com/r/visualization/comments/6yfa27/how_would_one_go_about_creating_a_stream_graph/)
Here's the tutorial book I used, if anyone was interested: http://shop.oreilly.com/product/0636920026938.do
After that was all done and dusted, the graph was exported into Adobe Illustrator for a bit of touching up (this is a bit like Adobe Photoshop, but for vector drawings).
Hope this helps!
I was in your boat at the start of last summer. I had a decently wide coding background, so I picked up JavaScript and made some visualizations using the threejs and d3 libraries. There are many d3 examples on https://bl.ocks.org/mbostock for you to use as a base if you go that route. If you only know R though, picking up JavaScript could take a bit of time.
This is great, thanks for sharing. You can really feel the effort that went into this. The final product itself is quite long, but those who have done these before know there is at least double the amount of notebook cells not shown that have been deleted and iterated on through its development.
One thing I haven't seen mentioned is adding a conclusions and lessons learned section at the end. Especially if you want to use this for a resume piece, employers really seem to like to see a discussion around the final findings you can pull from the analysis and how you developed from the experience. Additionally, having a future work section and how you imagine you can take the project further (ie. utilize the intuition you developed about the dataset to build a predictive win rate model) can highlight how you can conceptualize actionable products from your analysis. I also like to add a tool review about the key technologies I utilized and learned through the analysis to showcase how I developed those skills.
Separate from that, I think you'd benefit from adding d3.js to your toolbelt. It allows you to build any custom visualization you can dream up. It has a steep learning curve, but I don't think you'll have any trouble.
Thanks again for sharing, keep em coming!
-pH+
If you're going to be doing any data visualisation for a webpage, definitely checkout D3. It's a data visualisation API for JS. See the Link below.
Bit of a learning curve but there are some good youtube tutorials to start off with. I found this really great for a bioinformatics web application i helped develop a few months ago.
Oooh, this is what I'm into right now (for the web)!
The foundation for JS data visualization is D3.js, you can see some cool examples of visualizations with interaction on the bl.ocks website. I have recently been working closely with DC.js for interactive data-driven dashboards (you can play around with the dashboard on that link to see some powerful interactions). I am actually in the process of writing my own JS visualization library, so lmk if you have any questions about data-visualization in the web space.
Are you familiar with D3.js?
I worked as a programmer at a company where I would just translate excel documents into CSV with python and then run them through visualization JavaScript.
I would love if Wizards dumped a pile of spreadsheets on my desk!
Most server-side programming languages will let you capture the output of a shell command in one way or another - unfortunately it's unlikely you will get this functionality in a self-hosted web application with charts without some code, unless you stumble on an existing project on somewhere like github. You can certainly achieve this in the usual suspects (Python, Node, Java, Ruby etc). I'd start simple by just making a basic HTML template you can populate with the values from arp-scan, and iterate from there. If you're comfortable with Python, the first version of this could be made with Flask in not too many lines of code. When you get to the charting/graphs stage, I'd pick a nice client-side JavaScript library to draw them, something like d3.js.
Speaking of github, I'd also start searching there first for ideas - there's a good chance someone has had a crack at solving this problem well. The old 'pick the github project with the most stars' approach of software development usually works a treat.
This is a follow up to an OC I posted last week. I used this BLS table as the data source, and made the interactive with D3.
Check out cytoscape. This is essentially a solved problem. You also might want to have a look at d3.js since you get some really nice interactivity and there are already some pre-built graph visualizaiton tools in here.
Send me a PM. I had the opportunity to work on some graph visualization for my previous projects and might be able to help within reason or even under contract.
Typically a commercial software package like Tableau or Qlikview is used for dashboards like this. There might be some free/open source alternatives to these but I'm not as familiar with them.
Another option would be to go the d3 route or a charting library. This is Javascript programming whereas Tableau for example is a more tool-based, drag-n-drop workflow.
From my limited knowledge of Markov Chain (Just did a google image search) it appears to be a state diagram.
Would something like this be able to do it https://d3js.org/
If so I might be interested in doing it as I want to learn d3js
What you have looks really good. D3js can make it possible to present complex information and this example might be interesting to look at: https://bl.ocks.org/mbostock/4339083
If you click on the endpoints they collapse and expand allowing easy visualization.
Glad to hear you learned so much already! If you want to keep building on your project, I suggest you use this as an opportunity to learn d3, a javascript library that has a great stream graph implementation (and is just generally amazing for making interactive data visualizations) ;)
I am a total noob in WebDev, just starting to dip my feet in, so take this with a grain of salt.
https://d3js.org is a really cool framework that I've been toying with a tad. The thing that initially made me interested in it was this https://bost.ocks.org/mike/algorithms/ which a teacher showed me.
Some of the tutorials are a little out of date, if you try this and run into any problems send me a PM I would love to help figure it out.
I use d3.js for this. It's pretty impressive how much you get out of the box though some of the functionality is a little cryptic.
EDIT: Beale has only played the one match thus far, so he has a long row to hoe. He also gets bounced around the backline by Chieka so his ratings often get diluted across 10-15.
I used to do charts and graphs, largely in Excel believe it or not. Now, I work for a finance company. I'm not on the team, but we do have a graphics engineering dept.
I would encourage anyone interested in data visualization to explore the hell out of d3.js.
Also, The Wall Street Journal Guide to Information Graphics is worth reading.
The best data visualization experts have a remarkable ability to put themselves into the reader's frame of mind. They know to make axis fonts large enough to be readable and use easily divisible axes increments. Be aware of red-green colorblindness, and make sure to use both shapes and color to differentiate data series. Backgrounds often are best with neutral faded colors, I usually prefer a light tan-white fade from bottom to top. Background colors should have a slight bit of interest but never, under any circumstance, be distracting to the reader.
Cheers.
This seems like something like a nontechnical user would come up with. You know, when all you have is a hammer everything looks like a nail.
Sure you could do it, but why not just build a web front end? You have way more flexibility and control over what exactly you want to do. You also don't need to worry about the software on the local machines. There are also a ton of JavaScript libraries and templates out there that make this kind of stuff really easy and give you so many more features.
My lab uses python/matplotlib/seaborn (and then edited in Adobe Illustrator if need be) and D3.js for just about any figure we publish. Both can give you really great looking figures, but require some work to get it to look exactly how you want it.
Have a look at the Bar charts available in Plotly do some tests and see if it will produce something that will meet your needs.
Otherwise, if you want more granular control and nearly unlimited configurability, take a look at D3.js, and have your Python app be the backend feed data into D3.
Maybe take a look at D3: https://d3js.org/
This library handles large dataset animation efficiently.
To find more info talking about D3 and Svelte, try "amelia wattenberger svelte". Just an idea
Hi sorry I forgot to add initially. The data source was 200k rss feed provided by the podcasts that provide a summary of the podcast as well as most/all episodes with additional information for each including title, description, keywords, person identifiers, etc. The visualization data was crunched with php/mysql and the bar chart race itself was done with the D3.js library available at https://d3js.org/
Basically : no (?!?) - ok ok don't bite
Very opiniated answer : can you define "beautiful" ? Example : I find http://worrydream.com and https://d3js.org beautiful
Depending on your level of "interactivity", "reactivity", "low-level animation" we may have an idea of your javascript/css needs
It's like asking "do I need assembly to make a great game ?". Frameworks like Unity prove that you don't but at some point you're so fluent in OpenGL and Vertex shaders that you might as well know assembly :)
And to be clear : you will need to know / figure out / be able to represent / reason about what's going on under the hood - depending on your needs.
I think the short answer is no. Tableau is great for its purpose which is a business intelligence tool it is designed for something called VisQL or visual queries. i.e. complex data exploration and story telling in a business setting. You want something much more flexible and, hopefully, open source. I would encourage you to check out flourish (low/no code and paid) https://flourish.studio/features/ and d3js (code and open source) https://d3js.org
>- HTML/CSS/JS presentation: I don't need, and don't want, a fully-fledged, transcompiled, state-of-the-art , just-in-time javascript framework that brews coffee while rewriting my entire codebase. I will have just very simple overlays - but I'd like to animate them, have transitions etc. I want it to LOOK fancy, but be very basic. Normally I'd try to use jQuery, but there has to be something better by now.
https://d3js.org/ might be what you're looking for.
Yeah, it’s pretty odd to me how devs assume Flutter and RN are the only ways to build cross-platform native apps, and don’t consider other options.
I completely understand why one should be comfortable with picking up different langs to get the job done, as I’m doing it in my own projects. But it seems like most of the comments here are arguing why your [web-first] team should stick with a [non-web-first] toolset. A Flutter peg in a web hole. I’ve never been a tech lead, but I do know that it’s important to account for churn and what their team is good at. Web-wise, Flutter will always be playing catch-up, as the apps aren’t designed according to web standards. Flutter will always have to maintain its own layer of abstraction to make its apps compatible and performant on the web, just as it has to maintain its own native-looking UI components. It just seems like an anti-pattern to me.
I’ve never used Cordova, but apparently Capacitor solves Cordova’s issues by removing the abstraction layer managing the project directory, and following the native platform’s project structure: https://ionic.io/resources/articles/capacitor-vs-cordova-modern-hybrid-app-development
Tauri also avoids this limitation, and doesn’t bundle Chromium with every app too.
Lastly, I also hear that D3 is great for data viz.
Quite likely with D3.js.
There are similar (and even more complex) projects done by Shirley Wu, who also has a talk online on how she develops with ReactJS + D3js.
WebGL or a higher-level graphics library like D3.js (which uses WebGL behind the scenes). Alternatively you can cheat and pre-record a video with all the fancy animations, though in this case you would some interactivity.
You could use D3 for this. Here's an example:
https://bl.ocks.org/officeofjane/a70f4b44013d06b9c0a973f163d8ab7a
The animation is embedded in this article and was created with d3.js. The data comes from WHO/Europe.
About this post:
CSS can get into some interesting math once you want to start doing the neat things with it. For example, animation benefits significantly from understanding splines - https://tobiasahlin.com/blog/curved-path-animations-in-css/
You don't need it, but if you want to do some of the neat front end things - it becomes useful to have an idea of what is going on so that you can use the right tools to modify the math that is doing the graphics.
Everyone oohs and ahhs at the fancy interactive graphs and if you get into doing things with d3 or similar - you'll find math in it. https://d3js.org